[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:24 dist-test-slave-6zbq krb5kdc[1662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:24 dist-test-slave-6zbq krb5kdc[1662](info): set up 2 sockets
Aug 07 08:29:24 dist-test-slave-6zbq krb5kdc[1662](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 08:29:25.890475 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:25 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:26.005716 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43023
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:26.124707 1682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:26.124929 1682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:26.124982 1682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:26.128881 1682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:26.128929 1682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:26.128957 1682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:26.128978 1682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:26.129005 1682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:26.134513 1682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:26.135792 1682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:26.137001 1682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:26.149077 1689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:26.150640 1691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:26.150801 1688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:26.151352 1682 server_base.cc:1034] running on GCE node
I20260807 08:29:26.152030 1682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:26.153726 1682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:26.154887 1682 hybrid_clock.cc:648] HybridClock initialized: now 1786091366154861 us; error 59 us; skew 500 ppm
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:26.157617 1682 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:26.158795 1682 webserver.cc:460] Webserver started at http://127.1.157.126:38609/ using document root <none> and password file <none>
I20260807 08:29:26.159622 1682 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:26.159695 1682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:26.160115 1682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:26.162806 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "5a3cced99ec0433294d8b5875a96c809"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "3e4c7763096b34594527947e2cc071d9"
server_key_iv: "1c0ede9604132f514ddeac1440d52a4f"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.163734 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "5a3cced99ec0433294d8b5875a96c809"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "3e4c7763096b34594527947e2cc071d9"
server_key_iv: "1c0ede9604132f514ddeac1440d52a4f"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.168601 1682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 08:29:26.171868 1699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.173427 1682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:26.173611 1682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "5a3cced99ec0433294d8b5875a96c809"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "3e4c7763096b34594527947e2cc071d9"
server_key_iv: "1c0ede9604132f514ddeac1440d52a4f"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.173717 1682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:26.191637 1682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:26.195089 1682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:26.195286 1682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:26.209890 1682 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43023
I20260807 08:29:26.209942 1781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43023 every 8 connection(s)
I20260807 08:29:26.211210 1682 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:26.216399 1782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:26.221731 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1682
I20260807 08:29:26.221912 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:26.223753 1653 external_mini_cluster.cc:1428] Setting key 14665d4923411e736f0dbe5406ea5bf3
I20260807 08:29:26.227667 1782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809: Bootstrap starting.
I20260807 08:29:26.232483 1782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:26.234301 1782 log.cc:824] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:26.237108 1782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809: No bootstrap required, opened a new log
I20260807 08:29:26.243942 1782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } }
I20260807 08:29:26.244335 1782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:26.244436 1782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3cced99ec0433294d8b5875a96c809, State: Initialized, Role: FOLLOWER
I20260807 08:29:26.245224 1782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } }
I20260807 08:29:26.245399 1782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:26.245527 1782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:26.245656 1782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:26.246721 1782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } }
I20260807 08:29:26.247303 1782 leader_election.cc:302] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a3cced99ec0433294d8b5875a96c809; no voters:
I20260807 08:29:26.247658 1782 leader_election.cc:288] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:26.247781 1788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:26.248010 1788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [term 1 LEADER]: Becoming Leader. State: Replica: 5a3cced99ec0433294d8b5875a96c809, State: Running, Role: LEADER
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:26.248479 1788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } }
I20260807 08:29:26.249301 1782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:26.251542 1789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a3cced99ec0433294d8b5875a96c809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } } }
I20260807 08:29:26.251684 1789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:26.251618 1791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a3cced99ec0433294d8b5875a96c809. Latest consensus state: current_term: 1 leader_uuid: "5a3cced99ec0433294d8b5875a96c809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a3cced99ec0433294d8b5875a96c809" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43023 } } }
I20260807 08:29:26.251749 1791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:26.252441 1797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:26.258001 1797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:26.259145 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:26.229254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.1.157.126:43023)
0807 08:29:26.229810 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:26.229824 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:26.232717 (+ 2893us) server_negotiation.cc:407] Connection header received
0807 08:29:26.237635 (+ 4918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:26.237657 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:26.238136 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:26.238523 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:26.240841 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.242442 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.243749 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.244417 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.244597 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:26.249144 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:26.249175 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:26.249191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:26.249217 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:26.251624 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.252296 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.252304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.252310 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.252459 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.252884 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.252887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.252889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.253477 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:26.253656 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:26.254404 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 08:29:26.254667 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":193,"threads_started":1}
I20260807 08:29:26.267473 1797 catalog_manager.cc:1347] Generated new cluster ID: ae4fe36d106440a5861daec0a4127a8b
I20260807 08:29:26.267537 1797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:26.278482 1797 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:26.279573 1797 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:26.287187 1797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a3cced99ec0433294d8b5875a96c809: Generated new TSK 0
I20260807 08:29:26.288100 1797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:26.323653 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:26.467800 1822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:26.468026 1822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:26.468109 1822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:26.472084 1822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:26.472147 1822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:26.472222 1822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:26.477530 1822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:26.478542 1822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:26.479467 1822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:26.489508 1830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:26.489614 1832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:26.489743 1829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:26.490056 1822 server_base.cc:1034] running on GCE node
I20260807 08:29:26.490420 1822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:26.490962 1822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:26.492112 1822 hybrid_clock.cc:648] HybridClock initialized: now 1786091366492098 us; error 27 us; skew 500 ppm
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:26.495397 1822 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:26.496330 1822 webserver.cc:460] Webserver started at http://127.1.157.65:43919/ using document root <none> and password file <none>
I20260807 08:29:26.496798 1822 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:26.496851 1822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:26.497000 1822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:26.499063 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "00bd339c05d94df3a4b5737f66d2d3fe"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "5bc6bfe6820eb51f9e2cc3dc83226f86"
server_key_iv: "cd20a8bf9700bca0662501650ae827cb"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.499620 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "00bd339c05d94df3a4b5737f66d2d3fe"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "5bc6bfe6820eb51f9e2cc3dc83226f86"
server_key_iv: "cd20a8bf9700bca0662501650ae827cb"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.503367 1822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 08:29:26.505580 1844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.506702 1822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:26.506811 1822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "00bd339c05d94df3a4b5737f66d2d3fe"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "5bc6bfe6820eb51f9e2cc3dc83226f86"
server_key_iv: "cd20a8bf9700bca0662501650ae827cb"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.506894 1822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:26.528723 1822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:26.531366 1822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:26.531502 1822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:26.532199 1822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:26.533263 1822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:26.533311 1822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.533352 1822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:26.533380 1822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.555898 1822 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:40393
I20260807 08:29:26.556077 2012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:40393 every 8 connection(s)
I20260807 08:29:26.556715 1822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:26.560509 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1822
I20260807 08:29:26.560600 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:26.560833 1653 external_mini_cluster.cc:1428] Setting key 71ec95cca8249f35b406e9f6a90845ac
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:26.571302 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:26.558840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:58815 (local address 127.1.157.126:43023)
0807 08:29:26.558953 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:26.558956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:26.559884 (+ 928us) server_negotiation.cc:407] Connection header received
0807 08:29:26.561655 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:26.561658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:26.561711 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:26.561784 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:26.563145 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.563662 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.565315 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.565546 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.565626 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:26.568064 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:26.568067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:26.568069 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:26.568099 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:26.569401 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.569859 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.569862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.569863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.569938 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.570242 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.570245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.570247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.570451 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:26.570539 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:26.570999 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 08:29:26.571142 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:29:26.572032 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:26.559199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43023 (local address 127.1.157.65:58815)
0807 08:29:26.559755 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:26.559796 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:26.561459 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:26.561949 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:26.561956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:26.562461 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:26.562999 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:26.563015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.563777 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.563780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:26.565187 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:26.565199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.565699 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.565703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:26.565888 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:26.566467 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:26.566483 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:26.567917 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:26.569506 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:26.569511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:26.569524 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:26.569768 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:26.570019 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:26.570022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:26.570024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:26.570156 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:26.570549 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:26.570555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:26.570780 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 08:29:26.570935 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 08:29:26.571222 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":167,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:26.573176 2013 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43023
I20260807 08:29:26.573393 2013 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:26.573812 2013 heartbeater.cc:511] Master 127.1.157.126:43023 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:26.575855 1728 ts_manager.cc:194] Registered new tserver with Master: 00bd339c05d94df3a4b5737f66d2d3fe (127.1.157.65:40393)
I20260807 08:29:26.577598 1728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:58815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:26.589879 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:26.731748 2019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:26.731983 2019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:26.732044 2019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:26.736037 2019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:26.736083 2019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:26.736166 2019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:26.741376 2019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:26.742296 2019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:26.743264 2019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:26.753691 2025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:26.753667 2030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:26.753984 2019 server_base.cc:1034] running on GCE node
W20260807 08:29:26.754065 2027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:26.754479 2019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:26.755030 2019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:26.756199 2019 hybrid_clock.cc:648] HybridClock initialized: now 1786091366756177 us; error 49 us; skew 500 ppm
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:26.759372 2019 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:26.760123 2019 webserver.cc:460] Webserver started at http://127.1.157.66:39459/ using document root <none> and password file <none>
I20260807 08:29:26.760609 2019 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:26.760671 2019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:26.760828 2019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:26.763024 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "84e75852ce524fe1a166355cd02ab2c1"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "1fb3e377700421e91561786d28bd197c"
server_key_iv: "44b3780a94e28dd0b1af72ca3b820de5"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.763571 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "84e75852ce524fe1a166355cd02ab2c1"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "1fb3e377700421e91561786d28bd197c"
server_key_iv: "44b3780a94e28dd0b1af72ca3b820de5"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.767450 2019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:29:26.769771 2041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.770920 2019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:26.771010 2019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "84e75852ce524fe1a166355cd02ab2c1"
format_stamp: "Formatted at 2026-08-07 08:29:26 on dist-test-slave-6zbq"
server_key: "1fb3e377700421e91561786d28bd197c"
server_key_iv: "44b3780a94e28dd0b1af72ca3b820de5"
server_key_version: "encryptionkey@0"
I20260807 08:29:26.771088 2019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:26.784119 2019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:26.786911 2019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:26.787097 2019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:26.787735 2019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:26.788568 2019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:26.788612 2019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.788646 2019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:26.788667 2019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:26.809477 2019 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:40451
I20260807 08:29:26.809614 2208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:40451 every 8 connection(s)
I20260807 08:29:26.810446 2019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:26.816134 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2019
I20260807 08:29:26.816223 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:26.816445 1653 external_mini_cluster.cc:1428] Setting key 3599c95d5a2e0bc33f4b524702973356
Aug 07 08:29:26 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091366, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:26.827991 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:26.812342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40621 (local address 127.1.157.126:43023)
0807 08:29:26.812474 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:26.812478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:26.813471 (+ 993us) server_negotiation.cc:407] Connection header received
0807 08:29:26.815140 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:26.815143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:26.815190 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:26.815258 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:26.816622 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.817214 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.818590 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.818883 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.818990 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:26.824487 (+ 5497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:26.824492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:26.824495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:26.824524 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:26.825995 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.826486 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.826489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.826491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.826587 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:26.826940 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:26.826943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:26.826945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:26.827178 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:26.827279 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:26.827711 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 08:29:26.827827 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:29:26.828567 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:26.812691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43023 (local address 127.1.157.66:40621)
0807 08:29:26.813328 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:26.813369 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:26.814939 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:26.815398 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:26.815404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:26.815855 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:26.816477 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:26.816499 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.817331 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.817334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:26.818475 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:26.818486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:26.819019 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:26.819022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:26.819221 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:26.822759 (+ 3538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:26.822775 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:26.824325 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:26.826102 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:26.826107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:26.826118 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:26.826379 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:26.826670 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:26.826672 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:26.826674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:26.826829 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:26.827273 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:26.827277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:26.827474 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 08:29:26.827606 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 08:29:26.827823 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":203,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:26.829545 2211 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43023
I20260807 08:29:26.829742 2211 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:26.830152 2211 heartbeater.cc:511] Master 127.1.157.126:43023 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:29:26.831094 1728 ts_manager.cc:194] Registered new tserver with Master: 84e75852ce524fe1a166355cd02ab2c1 (127.1.157.66:40451)
I20260807 08:29:26.831805 1728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:40621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:26.846310 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:27.000190 2221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:27.000397 2221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:27.000442 2221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:27.004455 2221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:27.004514 2221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:27.004595 2221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:27.010447 2221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43023
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:27.011312 2221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:27.012204 2221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:27.022078 2228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:27.022109 2230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:27.022220 2221 server_base.cc:1034] running on GCE node
W20260807 08:29:27.022147 2227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:27.022719 2221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:27.023258 2221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:27.024403 2221 hybrid_clock.cc:648] HybridClock initialized: now 1786091367024374 us; error 45 us; skew 500 ppm
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:27.027524 2221 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:27.028304 2221 webserver.cc:460] Webserver started at http://127.1.157.67:39855/ using document root <none> and password file <none>
I20260807 08:29:27.028775 2221 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:27.028838 2221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:27.029004 2221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:27.031016 2221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "329ee5d2f8514e3585340995e741f595"
format_stamp: "Formatted at 2026-08-07 08:29:27 on dist-test-slave-6zbq"
server_key: "230de3d5cf409a2b6d792b031992c0ed"
server_key_iv: "da5d5e9e69101c3496c60508be91b60d"
server_key_version: "encryptionkey@0"
I20260807 08:29:27.031602 2221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "329ee5d2f8514e3585340995e741f595"
format_stamp: "Formatted at 2026-08-07 08:29:27 on dist-test-slave-6zbq"
server_key: "230de3d5cf409a2b6d792b031992c0ed"
server_key_iv: "da5d5e9e69101c3496c60508be91b60d"
server_key_version: "encryptionkey@0"
I20260807 08:29:27.035085 2221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:29:27.037263 2242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:27.038632 2221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:27.038740 2221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "329ee5d2f8514e3585340995e741f595"
format_stamp: "Formatted at 2026-08-07 08:29:27 on dist-test-slave-6zbq"
server_key: "230de3d5cf409a2b6d792b031992c0ed"
server_key_iv: "da5d5e9e69101c3496c60508be91b60d"
server_key_version: "encryptionkey@0"
I20260807 08:29:27.038843 2221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:27.058482 2221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:27.061004 2221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:27.061218 2221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:27.061818 2221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:27.062680 2221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:27.062736 2221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:27.062770 2221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:27.062790 2221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:27.083155 2221 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:43795
I20260807 08:29:27.083200 2415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:43795 every 8 connection(s)
I20260807 08:29:27.084018 2221 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:27.092996 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2221
I20260807 08:29:27.093119 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:27.093427 1653 external_mini_cluster.cc:1428] Setting key 0927c9ffe56ab0014753012933b8eac7
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:27.097476 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:27.085757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:60545 (local address 127.1.157.126:43023)
0807 08:29:27.085887 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:27.085891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:27.086799 (+ 908us) server_negotiation.cc:407] Connection header received
0807 08:29:27.088367 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:27.088370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:27.088416 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:27.088477 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:27.089824 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.090362 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.091570 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.091778 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.091844 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:27.094466 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:27.094470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:27.094473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:27.094498 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:27.095893 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.096342 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.096344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.096345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.096410 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.096644 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.096646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.096647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.096801 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:27.096868 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:27.097270 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 08:29:27.097366 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:29:27.098135 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:27.086107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43023 (local address 127.1.157.67:60545)
0807 08:29:27.086664 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:27.086706 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:27.088179 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:27.088614 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:27.088619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:27.089113 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:27.089713 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:27.089729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.090446 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.090449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:27.091459 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:27.091469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.091877 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.091879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:27.092070 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:27.092586 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:27.092601 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:27.094315 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:27.095989 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:27.095994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:27.096007 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:27.096266 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:27.096467 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:27.096469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:27.096470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:27.096584 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:27.096868 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:27.096872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:27.097087 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 08:29:27.097224 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 08:29:27.097439 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":111,"threads_started":1}
I20260807 08:29:27.099176 2416 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43023
I20260807 08:29:27.099376 2416 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:27.099787 2416 heartbeater.cc:511] Master 127.1.157.126:43023 requested a full tablet report, sending...
I20260807 08:29:27.100637 1728 ts_manager.cc:194] Registered new tserver with Master: 329ee5d2f8514e3585340995e741f595 (127.1.157.67:43795)
I20260807 08:29:27.101142 1728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:60545
I20260807 08:29:27.107430 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:27.134788 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:27.126601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.1.157.65:40393)
0807 08:29:27.126904 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:27.126910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:27.126933 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:29:27.127035 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:27.127040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:27.127211 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:27.127343 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:27.127937 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.128664 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.129443 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.129799 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.129887 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:27.132008 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:27.132018 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:27.132029 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:27.132053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:27.133344 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.133698 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.133703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.133704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.133777 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.133951 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.133953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.133954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.134160 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:27.134301 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:27.134479 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 08:29:27.134611 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260807 08:29:27.135844 1880 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39852
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[1662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:27.161835 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:27.154472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.1.157.65:40393)
0807 08:29:27.154596 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:27.154600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:27.154738 (+ 138us) server_negotiation.cc:407] Connection header received
0807 08:29:27.154849 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:27.154852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:27.154892 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:27.154949 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:27.155569 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.156035 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.156803 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:27.157016 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:27.157099 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:27.159185 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:27.159188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:27.159190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:27.159211 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:27.160258 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.160622 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.160626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.160627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.160701 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:27.160934 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:27.160937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:27.160939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:27.161099 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:27.161271 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:27.161571 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 08:29:27.161693 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:29:27.162781 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1822
I20260807 08:29:27.172747 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2019
I20260807 08:29:27.187497 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2221
I20260807 08:29:27.200433 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1682
2026-08-07T08:29:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2596 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[2435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[2435](info): set up 2 sockets
Aug 07 08:29:27 dist-test-slave-6zbq krb5kdc[2435](info): commencing operation
krb5kdc: starting...
W20260807 08:29:28.324031 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:28 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:28.432893 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43285
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:28.537582 2459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:28.537789 2459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:28.537829 2459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:28.541778 2459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:28.541828 2459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:28.541848 2459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:28.541878 2459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:28.541905 2459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:28.546876 2459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:28.547709 2459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:28.548568 2459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:28.556057 2470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:28.556059 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:28.556082 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:28.556154 2459 server_base.cc:1034] running on GCE node
I20260807 08:29:28.556821 2459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:28.557745 2459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:28.558894 2459 hybrid_clock.cc:648] HybridClock initialized: now 1786091368558873 us; error 31 us; skew 500 ppm
Aug 07 08:29:28 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:28.561313 2459 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:28.562037 2459 webserver.cc:460] Webserver started at http://127.1.157.126:34693/ using document root <none> and password file <none>
I20260807 08:29:28.562543 2459 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:28.562604 2459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:28.562789 2459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:28.564571 2459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "6268819690d240cb9fcf73359e420828"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "b2a4449a6b6dd0933c62c35fa3b5d6c2"
server_key_iv: "cc39476d8ad6da0b93176a6b83295148"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.565064 2459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "6268819690d240cb9fcf73359e420828"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "b2a4449a6b6dd0933c62c35fa3b5d6c2"
server_key_iv: "cc39476d8ad6da0b93176a6b83295148"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.568380 2459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:28.570626 2481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:28.571738 2459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:28.571844 2459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "6268819690d240cb9fcf73359e420828"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "b2a4449a6b6dd0933c62c35fa3b5d6c2"
server_key_iv: "cc39476d8ad6da0b93176a6b83295148"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.571939 2459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:28.593271 2459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:28.595577 2459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:28.595726 2459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:28.601867 2459 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43285
I20260807 08:29:28.601898 2560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43285 every 8 connection(s)
I20260807 08:29:28.602689 2459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:28.605793 2561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:28.607921 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2459
I20260807 08:29:28.608022 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:28.608250 1653 external_mini_cluster.cc:1428] Setting key 988e6eb04147fab91648e975899ffce8
I20260807 08:29:28.612298 2561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828: Bootstrap starting.
I20260807 08:29:28.614626 2561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:28.615451 2561 log.cc:824] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:28.617200 2561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828: No bootstrap required, opened a new log
Aug 07 08:29:28 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:28.620241 2561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } }
I20260807 08:29:28.620450 2561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:28.620484 2561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6268819690d240cb9fcf73359e420828, State: Initialized, Role: FOLLOWER
I20260807 08:29:28.620873 2561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } }
I20260807 08:29:28.621002 2561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:28.621063 2561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:28.621177 2561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:28.621901 2561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } }
I20260807 08:29:28.622218 2561 leader_election.cc:302] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6268819690d240cb9fcf73359e420828; no voters:
I20260807 08:29:28.622511 2561 leader_election.cc:288] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:28.622663 2568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:28.622849 2568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [term 1 LEADER]: Becoming Leader. State: Replica: 6268819690d240cb9fcf73359e420828, State: Running, Role: LEADER
I20260807 08:29:28.623131 2568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } }
I20260807 08:29:28.623522 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:28.610326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.1.157.126:43285)
0807 08:29:28.610831 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:28.610850 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:28.610892 (+ 42us) server_negotiation.cc:407] Connection header received
0807 08:29:28.611610 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:28.611628 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:28.611974 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:28.612338 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:28.613155 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.614083 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.615233 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.615710 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.615853 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:28.618257 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:28.618262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:28.618274 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:28.618298 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:28.620727 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:28.621189 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:28.621195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:28.621201 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:28.621331 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:28.621591 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:28.621594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:28.621596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:28.621935 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:28.622109 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:28.622414 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 08:29:28.622694 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":127,"threads_started":1}
I20260807 08:29:28.623665 2561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:28.624734 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6268819690d240cb9fcf73359e420828. Latest consensus state: current_term: 1 leader_uuid: "6268819690d240cb9fcf73359e420828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } } }
I20260807 08:29:28.624856 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:28.625136 2571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6268819690d240cb9fcf73359e420828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6268819690d240cb9fcf73359e420828" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43285 } } }
I20260807 08:29:28.625238 2571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:28.625576 2578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:28.628468 2578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:28.633677 2578 catalog_manager.cc:1347] Generated new cluster ID: c9b0e662c8f64da7b51eb8cf282b7e50
I20260807 08:29:28.633728 2578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:28.645643 2578 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:28.646456 2578 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:28.651391 2578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6268819690d240cb9fcf73359e420828: Generated new TSK 0
I20260807 08:29:28.651895 2578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:28.687047 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:28.826272 2603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:28.826506 2603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:28.826551 2603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:28.830452 2603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:28.830498 2603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:28.830565 2603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:28.835461 2603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:28.836314 2603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:28.837168 2603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:28.845765 2613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:28.845782 2614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:28.845808 2603 server_base.cc:1034] running on GCE node
W20260807 08:29:28.845968 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:28.846328 2603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:28.846915 2603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:28.848073 2603 hybrid_clock.cc:648] HybridClock initialized: now 1786091368848051 us; error 33 us; skew 500 ppm
Aug 07 08:29:28 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:28.850610 2603 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:28.851367 2603 webserver.cc:460] Webserver started at http://127.1.157.65:43727/ using document root <none> and password file <none>
I20260807 08:29:28.851843 2603 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:28.851900 2603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:28.852078 2603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:28.853922 2603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "bda9fdde44634d54a8022d9a06d53b55"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "a01c071ebe23bfdd932213331a7ff954"
server_key_iv: "baa5e9827522596d23866b9b12b2ccde"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.854471 2603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "bda9fdde44634d54a8022d9a06d53b55"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "a01c071ebe23bfdd932213331a7ff954"
server_key_iv: "baa5e9827522596d23866b9b12b2ccde"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.857743 2603 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:28.859912 2626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:28.860973 2603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:28.861089 2603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "bda9fdde44634d54a8022d9a06d53b55"
format_stamp: "Formatted at 2026-08-07 08:29:28 on dist-test-slave-6zbq"
server_key: "a01c071ebe23bfdd932213331a7ff954"
server_key_iv: "baa5e9827522596d23866b9b12b2ccde"
server_key_version: "encryptionkey@0"
I20260807 08:29:28.861182 2603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:28.876302 2603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:28.878849 2603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:28.879009 2603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:28.879514 2603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:28.880270 2603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:28.880316 2603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:28.880365 2603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:28.880394 2603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:28.888402 2603 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:42617
I20260807 08:29:28.888468 2795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:42617 every 8 connection(s)
I20260807 08:29:28.889204 2603 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:28.892805 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2603
I20260807 08:29:28.892900 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:28.893180 1653 external_mini_cluster.cc:1428] Setting key 8a362d34940995f7b90839193055d37e
Aug 07 08:29:28 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:28.902834 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:28.890836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40881 (local address 127.1.157.126:43285)
0807 08:29:28.890967 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:28.890971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:28.891725 (+ 754us) server_negotiation.cc:407] Connection header received
0807 08:29:28.893066 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:28.893070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:28.893127 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:28.893233 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:28.894601 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.895331 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.896440 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.896716 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.896794 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:28.899495 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:28.899498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:28.899500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:28.899523 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:28.900766 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:28.901343 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:28.901345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:28.901346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:28.901413 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:28.901753 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:28.901755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:28.901756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:28.901910 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:28.901979 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:28.902580 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 08:29:28.902683 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:28.903793 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:28.891166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43285 (local address 127.1.157.65:40881)
0807 08:29:28.891588 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:28.891628 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:28.892820 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:28.893390 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:28.893396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:28.893803 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:28.894448 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:28.894470 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.895442 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.895444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:28.896304 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:28.896316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:28.896853 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:28.896856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:28.897051 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:28.897601 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:28.897621 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:28.899336 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:28.900877 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:28.900883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:28.900898 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:28.901249 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:28.901490 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:28.901494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:28.901495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:28.901672 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:28.902004 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:28.902010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:28.902310 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 08:29:28.902557 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 08:29:28.902878 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260807 08:29:28.904891 2796 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43285
I20260807 08:29:28.905120 2796 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:28.905570 2796 heartbeater.cc:511] Master 127.1.157.126:43285 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:28.907163 2503 ts_manager.cc:194] Registered new tserver with Master: bda9fdde44634d54a8022d9a06d53b55 (127.1.157.65:42617)
I20260807 08:29:28.908268 2503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:40881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:28.918489 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:29.023540 2806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:29.023749 2806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:29.023794 2806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:29.027694 2806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:29.027741 2806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:29.027801 2806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:29.032711 2806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:29.033555 2806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:29.034457 2806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:29.043134 2820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:29.043114 2815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:29.043105 2813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:29.043711 2806 server_base.cc:1034] running on GCE node
I20260807 08:29:29.044109 2806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:29.044778 2806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:29.045981 2806 hybrid_clock.cc:648] HybridClock initialized: now 1786091369045943 us; error 52 us; skew 500 ppm
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:29.048590 2806 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:29.049336 2806 webserver.cc:460] Webserver started at http://127.1.157.66:43843/ using document root <none> and password file <none>
I20260807 08:29:29.049811 2806 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:29.049872 2806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:29.050037 2806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:29.051934 2806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "6d0489fe20d646319a2eb4b11b4fd03d"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "54dc2b3380ee8d62209fdf9aa7851b66"
server_key_iv: "6bdfcc514deaeb63ae5688cfc7143a97"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.052451 2806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "6d0489fe20d646319a2eb4b11b4fd03d"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "54dc2b3380ee8d62209fdf9aa7851b66"
server_key_iv: "6bdfcc514deaeb63ae5688cfc7143a97"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.055795 2806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 08:29:29.057861 2831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.058956 2806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:29.059056 2806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "6d0489fe20d646319a2eb4b11b4fd03d"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "54dc2b3380ee8d62209fdf9aa7851b66"
server_key_iv: "6bdfcc514deaeb63ae5688cfc7143a97"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.059147 2806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:29.080458 2806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:29.083258 2806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:29.083412 2806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:29.083909 2806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:29.084638 2806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:29.084685 2806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.084734 2806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:29.084762 2806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.092865 2806 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37159
I20260807 08:29:29.092895 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37159 every 8 connection(s)
I20260807 08:29:29.093664 2806 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:29.103940 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2806
I20260807 08:29:29.104063 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:29.104331 1653 external_mini_cluster.cc:1428] Setting key 7ef60119aac4a7480ab5f5b08daf314c
I20260807 08:29:29.105818 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.095241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58809 (local address 127.1.157.126:43285)
0807 08:29:29.095354 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.095358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.096092 (+ 734us) server_negotiation.cc:407] Connection header received
0807 08:29:29.097328 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.097331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.097406 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.097474 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.098528 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.099051 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.100117 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.100338 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.100424 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.102514 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.102517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.102520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.102544 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.103623 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.104239 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.104242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.104243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.104322 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.104683 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.104686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.104688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.105037 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.105136 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.105578 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.105707 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:29.106789 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.095544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43285 (local address 127.1.157.66:58809)
0807 08:29:29.095960 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.095998 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.097141 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.097655 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.097661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.097956 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:29.098399 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.098416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.099166 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.099169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.099997 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.100006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.100468 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.100472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.100630 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.101034 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:29.101048 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:29.102371 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:29.103743 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:29.103749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:29.103769 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:29.104093 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:29.104418 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:29.104421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:29.104423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:29.104594 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:29.105138 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:29.105143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:29.105344 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.105515 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.105810 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260807 08:29:29.107908 3015 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43285
I20260807 08:29:29.108105 3015 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:29.108516 3015 heartbeater.cc:511] Master 127.1.157.126:43285 requested a full tablet report, sending...
I20260807 08:29:29.109423 2503 ts_manager.cc:194] Registered new tserver with Master: 6d0489fe20d646319a2eb4b11b4fd03d (127.1.157.66:37159)
I20260807 08:29:29.110121 2503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:58809
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:29.128165 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:29.232000 3020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:29.232206 3020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:29.232249 3020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:29.236095 3020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:29.236141 3020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:29.236210 3020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:29.241039 3020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:29.241925 3020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:29.242848 3020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:29.251374 3032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:29.251411 3027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:29.251386 3030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:29.251725 3020 server_base.cc:1034] running on GCE node
I20260807 08:29:29.252104 3020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:29.252645 3020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:29.253800 3020 hybrid_clock.cc:648] HybridClock initialized: now 1786091369253783 us; error 34 us; skew 500 ppm
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:29.256237 3020 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:29.256960 3020 webserver.cc:460] Webserver started at http://127.1.157.67:37435/ using document root <none> and password file <none>
I20260807 08:29:29.257407 3020 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:29.257458 3020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:29.257609 3020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:29.259424 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "df637222a5e5430a8d9c629c41dffbcd"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "6c4738773abe3b16ed2d5866fc75ac85"
server_key_iv: "df7e948087e73ab12bf265e026cb4904"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.259905 3020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "df637222a5e5430a8d9c629c41dffbcd"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "6c4738773abe3b16ed2d5866fc75ac85"
server_key_iv: "df7e948087e73ab12bf265e026cb4904"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.263518 3020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:29.265705 3041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.266768 3020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:29.266878 3020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "df637222a5e5430a8d9c629c41dffbcd"
format_stamp: "Formatted at 2026-08-07 08:29:29 on dist-test-slave-6zbq"
server_key: "6c4738773abe3b16ed2d5866fc75ac85"
server_key_iv: "df7e948087e73ab12bf265e026cb4904"
server_key_version: "encryptionkey@0"
I20260807 08:29:29.266963 3020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:29.279767 3020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:29.282097 3020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:29.282249 3020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:29.282768 3020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:29.283506 3020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:29.283551 3020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.283600 3020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:29.283630 3020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:29.293268 3020 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:42553
I20260807 08:29:29.293309 3204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:42553 every 8 connection(s)
I20260807 08:29:29.294046 3020 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:29.303498 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3020
I20260807 08:29:29.303615 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:29.303853 1653 external_mini_cluster.cc:1428] Setting key 466d125d1094113cc707724cd65f86af
I20260807 08:29:29.307488 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.295701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:47793 (local address 127.1.157.126:43285)
0807 08:29:29.295831 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.295834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.296558 (+ 724us) server_negotiation.cc:407] Connection header received
0807 08:29:29.297776 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.297780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.297829 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.297902 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.298992 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.299528 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.300659 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.300902 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.301000 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.303832 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.303835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.303838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.303865 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.305480 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.306036 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.306039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.306040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.306119 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.306453 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.306457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.306459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.306642 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.306741 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.307210 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.307337 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:29:29.308166 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.295981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43285 (local address 127.1.157.67:47793)
0807 08:29:29.296396 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.296437 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.297563 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.298041 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.298047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.298409 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:29.298868 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.298881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.299648 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.299651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.300510 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.300520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.301068 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.301071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.301373 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.301947 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:29.301963 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:29.303645 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:29.305593 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:29.305599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:29.305612 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:29.305940 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:29.306207 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:29.306210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:29.306212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:29.306372 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:29.306751 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:29.306757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:29.306979 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.307127 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.307432 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260807 08:29:29.309208 3205 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43285
I20260807 08:29:29.309405 3205 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:29.309823 3205 heartbeater.cc:511] Master 127.1.157.126:43285 requested a full tablet report, sending...
I20260807 08:29:29.310889 2503 ts_manager.cc:194] Registered new tserver with Master: df637222a5e5430a8d9c629c41dffbcd (127.1.157.67:42553)
I20260807 08:29:29.311411 2503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:47793
I20260807 08:29:29.317436 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:29:29.327615 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.321350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.1.157.126:43285)
0807 08:29:29.321475 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.321479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.321597 (+ 118us) server_negotiation.cc:407] Connection header received
0807 08:29:29.321696 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.321698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.321731 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.321790 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.322371 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.322851 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.323660 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.323896 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.323965 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.325085 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.325088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.325090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.325110 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.326293 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.326668 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.326678 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.326679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.326746 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.326969 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.326971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.326972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.327156 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.327254 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.327410 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.327489 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:29:29.332888 2503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:29.336149 2503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:29.351051 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.345485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42617 (local address 127.0.0.1:52740)
0807 08:29:29.346506 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.346518 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.346679 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.347137 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.347143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.347165 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:29.347293 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.347300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348371 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.348374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.349661 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.349671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.350559 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.350562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.350671 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.350708 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.350801 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.350913 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":205,"thread_start_us":102,"threads_started":1}
I20260807 08:29:29.351435 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.345379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42553 (local address 127.0.0.1:38018)
0807 08:29:29.346290 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.346332 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.346541 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.347228 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.347230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.347247 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:29.347339 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.347345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348755 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.348759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.350035 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.350047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.351061 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.351065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.351156 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.351179 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.351241 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.351309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":30,"thread_start_us":65,"threads_started":1}
I20260807 08:29:29.351511 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.345508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.1.157.67:42553)
0807 08:29:29.346054 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.346065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.346438 (+ 373us) server_negotiation.cc:407] Connection header received
0807 08:29:29.346714 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.346720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.346913 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.347034 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:29.347455 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348595 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.350285 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.350942 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.351039 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.351241 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.351290 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.351375 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":148,"threads_started":1}
I20260807 08:29:29.351809 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.345850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.1.157.65:42617)
0807 08:29:29.346267 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.346278 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.346611 (+ 333us) server_negotiation.cc:407] Connection header received
0807 08:29:29.346853 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.346857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.346995 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.347126 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:29.347396 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348251 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.349783 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.350422 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.351231 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.351370 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.351418 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.351497 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":140,"threads_started":1}
I20260807 08:29:29.352510 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.346019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37159 (local address 127.0.0.1:38718)
0807 08:29:29.346620 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.346634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.346722 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.347414 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.347417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.347435 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:29.347508 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.347513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348804 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.348807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.350035 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.350047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.351788 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.351792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.352208 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.352231 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.352315 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.352385 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":44,"thread_start_us":85,"threads_started":1}
I20260807 08:29:29.352991 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.346443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38718 (local address 127.1.157.66:37159)
0807 08:29:29.346991 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.347003 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.347024 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:29:29.347093 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.347098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.347269 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.347413 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:29.347610 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.348676 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.350222 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.351031 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.352020 (+ 989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.352148 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.352679 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.352761 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":138,"threads_started":1}
I20260807 08:29:29.354522 3103 tablet_service.cc:1467] Processing CreateTablet for tablet 00e0dd2e0d6c4da9b63a46dd7b768545 (DEFAULT_TABLE table=test-table [id=1e79e6c144de4721913a2ba800bde9bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:29.355517 3103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e0dd2e0d6c4da9b63a46dd7b768545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:29.355432 2914 tablet_service.cc:1467] Processing CreateTablet for tablet 00e0dd2e0d6c4da9b63a46dd7b768545 (DEFAULT_TABLE table=test-table [id=1e79e6c144de4721913a2ba800bde9bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:29.356391 2914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e0dd2e0d6c4da9b63a46dd7b768545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:29.356215 2693 tablet_service.cc:1467] Processing CreateTablet for tablet 00e0dd2e0d6c4da9b63a46dd7b768545 (DEFAULT_TABLE table=test-table [id=1e79e6c144de4721913a2ba800bde9bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:29.357177 2693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e0dd2e0d6c4da9b63a46dd7b768545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:29.363009 3235 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: Bootstrap starting.
I20260807 08:29:29.364401 3236 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: Bootstrap starting.
I20260807 08:29:29.366534 3235 tablet_bootstrap.cc:654] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:29.366724 3237 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Bootstrap starting.
I20260807 08:29:29.367723 3236 tablet_bootstrap.cc:654] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:29.368108 3235 log.cc:824] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:29.369127 3236 log.cc:824] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:29.369683 3237 tablet_bootstrap.cc:654] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:29.370508 3235 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: No bootstrap required, opened a new log
I20260807 08:29:29.370723 3235 ts_tablet_manager.cc:1397] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260807 08:29:29.371134 3236 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: No bootstrap required, opened a new log
I20260807 08:29:29.371317 3236 ts_tablet_manager.cc:1397] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:29:29.371990 3237 log.cc:824] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:29.374203 3237 tablet_bootstrap.cc:492] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: No bootstrap required, opened a new log
I20260807 08:29:29.374469 3237 ts_tablet_manager.cc:1397] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 08:29:29.375918 3236 raft_consensus.cc:348] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.376214 3236 raft_consensus.cc:374] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:29.376277 3236 raft_consensus.cc:729] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df637222a5e5430a8d9c629c41dffbcd, State: Initialized, Role: FOLLOWER
I20260807 08:29:29.376667 3236 consensus_queue.cc:260] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.376832 3235 raft_consensus.cc:348] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.377095 3235 raft_consensus.cc:374] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:29.377146 3235 raft_consensus.cc:729] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0489fe20d646319a2eb4b11b4fd03d, State: Initialized, Role: FOLLOWER
I20260807 08:29:29.377645 3236 ts_tablet_manager.cc:1428] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260807 08:29:29.377604 3235 consensus_queue.cc:260] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.378026 3205 heartbeater.cc:503] Master 127.1.157.126:43285 was elected leader, sending a full tablet report...
I20260807 08:29:29.378408 3015 heartbeater.cc:503] Master 127.1.157.126:43285 was elected leader, sending a full tablet report...
I20260807 08:29:29.378566 3235 ts_tablet_manager.cc:1428] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260807 08:29:29.380057 3237 raft_consensus.cc:348] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.380256 3237 raft_consensus.cc:374] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:29.380295 3237 raft_consensus.cc:729] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda9fdde44634d54a8022d9a06d53b55, State: Initialized, Role: FOLLOWER
I20260807 08:29:29.380580 3237 consensus_queue.cc:260] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.381260 2796 heartbeater.cc:503] Master 127.1.157.126:43285 was elected leader, sending a full tablet report...
I20260807 08:29:29.382987 3237 ts_tablet_manager.cc:1428] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
W20260807 08:29:29.390269 2798 tablet.cc:2369] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:29.400703 3244 raft_consensus.cc:482] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:29.400854 3244 raft_consensus.cc:504] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.401856 3244 leader_election.cc:288] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df637222a5e5430a8d9c629c41dffbcd (127.1.157.67:42553), 6d0489fe20d646319a2eb4b11b4fd03d (127.1.157.66:37159)
I20260807 08:29:29.405790 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.402162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52345 (local address 127.1.157.67:42553)
0807 08:29:29.402305 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.402310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.402326 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:29:29.402476 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.402479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.402526 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.402602 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:29.402998 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.403588 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.404828 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.405280 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.405366 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.405403 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.405649 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.405696 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 08:29:29.405812 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.402162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42553 (local address 127.1.157.65:52345)
0807 08:29:29.402269 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.402281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.402368 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.402629 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.402632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.402652 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:29.402885 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.402895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.403695 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.403697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.404695 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.404708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.405391 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.405395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.405538 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.405563 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.405635 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.405697 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 08:29:29.406695 3131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e0dd2e0d6c4da9b63a46dd7b768545" candidate_uuid: "bda9fdde44634d54a8022d9a06d53b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df637222a5e5430a8d9c629c41dffbcd" is_pre_election: true
I20260807 08:29:29.407042 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.402690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:58301 (local address 127.1.157.66:37159)
0807 08:29:29.402819 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.402823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.403075 (+ 252us) server_negotiation.cc:407] Connection header received
0807 08:29:29.403161 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.403164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.403209 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.403284 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:29.403628 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.404100 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.405886 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.406423 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.406513 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.406545 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.406900 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.406960 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:29:29.407114 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.402528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37159 (local address 127.1.157.65:58301)
0807 08:29:29.402951 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:29.402963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:29.403084 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:29.403292 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:29.403294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:29.403305 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:29.403535 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.403541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.404205 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.404209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.405765 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:29.405774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.406623 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.406629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:29.406769 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.406795 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:29:29.406883 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 08:29:29.406951 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":184,"threads_started":1}
I20260807 08:29:29.407085 3131 raft_consensus.cc:2454] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda9fdde44634d54a8022d9a06d53b55 in term 0.
I20260807 08:29:29.407706 2627 leader_election.cc:302] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bda9fdde44634d54a8022d9a06d53b55, df637222a5e5430a8d9c629c41dffbcd; no voters:
I20260807 08:29:29.407855 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e0dd2e0d6c4da9b63a46dd7b768545" candidate_uuid: "bda9fdde44634d54a8022d9a06d53b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" is_pre_election: true
I20260807 08:29:29.408098 2945 raft_consensus.cc:2454] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bda9fdde44634d54a8022d9a06d53b55 in term 0.
I20260807 08:29:29.408380 3244 raft_consensus.cc:2781] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:29.408465 3244 raft_consensus.cc:482] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:29.408517 3244 raft_consensus.cc:3037] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:29.409874 3244 raft_consensus.cc:504] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.410135 3244 leader_election.cc:288] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [CANDIDATE]: Term 1 election: Requested vote from peers df637222a5e5430a8d9c629c41dffbcd (127.1.157.67:42553), 6d0489fe20d646319a2eb4b11b4fd03d (127.1.157.66:37159)
I20260807 08:29:29.410465 3131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e0dd2e0d6c4da9b63a46dd7b768545" candidate_uuid: "bda9fdde44634d54a8022d9a06d53b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df637222a5e5430a8d9c629c41dffbcd"
I20260807 08:29:29.410573 2945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e0dd2e0d6c4da9b63a46dd7b768545" candidate_uuid: "bda9fdde44634d54a8022d9a06d53b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0489fe20d646319a2eb4b11b4fd03d"
I20260807 08:29:29.410580 3131 raft_consensus.cc:3037] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:29.410676 2945 raft_consensus.cc:3037] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:29.412573 3131 raft_consensus.cc:2454] T 00e0dd2e0d6c4da9b63a46dd7b768545 P df637222a5e5430a8d9c629c41dffbcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda9fdde44634d54a8022d9a06d53b55 in term 1.
I20260807 08:29:29.412792 2945 raft_consensus.cc:2454] T 00e0dd2e0d6c4da9b63a46dd7b768545 P 6d0489fe20d646319a2eb4b11b4fd03d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda9fdde44634d54a8022d9a06d53b55 in term 1.
I20260807 08:29:29.412875 2627 leader_election.cc:302] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bda9fdde44634d54a8022d9a06d53b55, df637222a5e5430a8d9c629c41dffbcd; no voters:
I20260807 08:29:29.413020 3244 raft_consensus.cc:2781] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:29.413211 3244 raft_consensus.cc:686] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [term 1 LEADER]: Becoming Leader. State: Replica: bda9fdde44634d54a8022d9a06d53b55, State: Running, Role: LEADER
I20260807 08:29:29.413486 3244 consensus_queue.cc:237] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } }
I20260807 08:29:29.416231 2500 catalog_manager.cc:5495] T 00e0dd2e0d6c4da9b63a46dd7b768545 P bda9fdde44634d54a8022d9a06d53b55 reported cstate change: term changed from 0 to 1, leader changed from <none> to bda9fdde44634d54a8022d9a06d53b55 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "bda9fdde44634d54a8022d9a06d53b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df637222a5e5430a8d9c629c41dffbcd" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bda9fdde44634d54a8022d9a06d53b55" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0489fe20d646319a2eb4b11b4fd03d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37159 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:29.433344 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.424380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.1.157.65:42617)
0807 08:29:29.424473 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.424476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.424806 (+ 330us) server_negotiation.cc:407] Connection header received
0807 08:29:29.424891 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.424894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.424932 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.424992 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.425827 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.426328 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.427412 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.427621 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.427701 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.430136 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.430145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.430155 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.430176 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.431817 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.432202 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.432207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.432209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.432284 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.432520 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.432523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.432525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.432745 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.432920 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.433114 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.433217 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:29.461465 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.454306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.1.157.65:42617)
0807 08:29:29.454432 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.454437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.454569 (+ 132us) server_negotiation.cc:407] Connection header received
0807 08:29:29.454677 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.454680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.454718 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.454791 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.455420 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.455885 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.456652 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.456864 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.456948 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.459135 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.459138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.459140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.459162 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.460247 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.460580 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.460582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.460583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.460644 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.460827 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.460829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.460830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.460963 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.461144 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.461247 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.461333 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:29.486362 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:29.479322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.1.157.65:42617)
0807 08:29:29.479436 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:29.479439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:29.479561 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:29:29.479664 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:29.479667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:29.479703 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:29.479763 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:29.480346 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.480790 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.481571 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:29.481772 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:29.481853 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:29.483862 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:29.483866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:29.483868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:29.483888 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:29.484945 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.485307 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.485310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.485311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.485380 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:29.485612 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:29.485615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:29.485617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:29.485763 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:29.485931 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:29.486103 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:29:29.486198 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 08:29:29.487792 2668 tablet_service.cc:2908] Rejecting scan request for tablet 00e0dd2e0d6c4da9b63a46dd7b768545: Uninitialized: clean time has not yet been initialized
I20260807 08:29:29.490203 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2603
I20260807 08:29:29.504665 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2806
I20260807 08:29:29.513684 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3020
I20260807 08:29:29.534029 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2459
2026-08-07T08:29:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2336 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[3267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[3267](info): set up 2 sockets
Aug 07 08:29:29 dist-test-slave-6zbq krb5kdc[3267](info): commencing operation
krb5kdc: starting...
W20260807 08:29:30.660349 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:30 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:30.767127 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34005
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:34005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:30.871803 3287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:30.872012 3287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:30.872051 3287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:30.875849 3287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:30.875895 3287 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:29:30.875910 3287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:30.875939 3287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:30.875969 3287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:30.875994 3287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:30.881227 3287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:34005
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:30.882079 3287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:30.882982 3287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:29:30.890558 3287 server_base.cc:1034] running on GCE node
W20260807 08:29:30.890481 3294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:30.890509 3299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:30.890496 3293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:30.891372 3287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:30.892343 3287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:30.893491 3287 hybrid_clock.cc:648] HybridClock initialized: now 1786091370893471 us; error 35 us; skew 500 ppm
Aug 07 08:29:30 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091370, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:30.896003 3287 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:30.896824 3287 webserver.cc:460] Webserver started at http://127.1.157.126:33267/ using document root <none> and password file <none>
I20260807 08:29:30.897311 3287 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:30.897375 3287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:30.897543 3287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:30.899430 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "dcc4d42f8d62495c813dcdb60c409425"
format_stamp: "Formatted at 2026-08-07 08:29:30 on dist-test-slave-6zbq"
server_key: "2fd23341734b31bc94556fcb1dd98fb4"
server_key_iv: "60f59a2790c2013bf918875d130f865e"
server_key_version: "encryptionkey@0"
I20260807 08:29:30.899951 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "dcc4d42f8d62495c813dcdb60c409425"
format_stamp: "Formatted at 2026-08-07 08:29:30 on dist-test-slave-6zbq"
server_key: "2fd23341734b31bc94556fcb1dd98fb4"
server_key_iv: "60f59a2790c2013bf918875d130f865e"
server_key_version: "encryptionkey@0"
I20260807 08:29:30.903376 3287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 08:29:30.905458 3310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:30.906582 3287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 08:29:30.906706 3287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "dcc4d42f8d62495c813dcdb60c409425"
format_stamp: "Formatted at 2026-08-07 08:29:30 on dist-test-slave-6zbq"
server_key: "2fd23341734b31bc94556fcb1dd98fb4"
server_key_iv: "60f59a2790c2013bf918875d130f865e"
server_key_version: "encryptionkey@0"
I20260807 08:29:30.906792 3287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:30.938822 3287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:30.941744 3287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:30.941910 3287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:30.949505 3287 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:34005
I20260807 08:29:30.949518 3408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:34005 every 8 connection(s)
I20260807 08:29:30.950332 3287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:30.952055 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3287
I20260807 08:29:30.952159 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:30.952387 1653 external_mini_cluster.cc:1428] Setting key 05f8196b59611b96be7f45e137f3a59e
I20260807 08:29:30.953128 3409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:30.959525 3409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Bootstrap starting.
Aug 07 08:29:30 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:30.961792 3409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:30.963195 3409 log.cc:824] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:30.964931 3409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: No bootstrap required, opened a new log
I20260807 08:29:30.965857 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:30.953659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.1.157.126:34005)
0807 08:29:30.954055 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:30.954073 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:30.954114 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:29:30.954936 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:30.954958 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:30.955340 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:30.955683 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:30.956298 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:30.957723 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:30.958533 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:30.959196 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:30.959383 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:30.961500 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:30.961506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:30.961520 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:30.961550 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:30.963410 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:30.963798 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:30.963803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:30.963807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:30.963915 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:30.964160 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:30.964163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:30.964165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:30.964512 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:30.964689 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:30.964946 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:29:30.965225 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":108,"threads_started":1}
I20260807 08:29:30.968076 3409 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } }
I20260807 08:29:30.968290 3409 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:30.968356 3409 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc4d42f8d62495c813dcdb60c409425, State: Initialized, Role: FOLLOWER
I20260807 08:29:30.968714 3409 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } }
I20260807 08:29:30.968837 3409 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:30.968900 3409 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:30.968993 3409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:30.969748 3409 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } }
I20260807 08:29:30.970043 3409 leader_election.cc:302] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc4d42f8d62495c813dcdb60c409425; no voters:
I20260807 08:29:30.970328 3409 leader_election.cc:288] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:30.970436 3415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:30.970624 3415 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [term 1 LEADER]: Becoming Leader. State: Replica: dcc4d42f8d62495c813dcdb60c409425, State: Running, Role: LEADER
I20260807 08:29:30.970906 3415 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } }
I20260807 08:29:30.971421 3409 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:30.972337 3418 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc4d42f8d62495c813dcdb60c409425. Latest consensus state: current_term: 1 leader_uuid: "dcc4d42f8d62495c813dcdb60c409425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } } }
I20260807 08:29:30.972369 3416 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc4d42f8d62495c813dcdb60c409425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4d42f8d62495c813dcdb60c409425" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34005 } } }
I20260807 08:29:30.972466 3418 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:30.972466 3416 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:30.972846 3428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:30.975730 3428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:30.980788 3428 catalog_manager.cc:1347] Generated new cluster ID: f8696018b8754c778cf3c1bcd2c8af1d
I20260807 08:29:30.980847 3428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:31.001339 3428 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:31.002429 3428 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:31.011814 3428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Generated new TSK 0
I20260807 08:29:31.012387 3428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:31.040809 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34005
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:31.174302 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:31.174521 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:31.174561 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:31.178382 3445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:31.178426 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:31.178460 3445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:31.178496 3445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:31.183277 3445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:31.184048 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:31.184861 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:31.193449 3456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:31.193455 3462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:31.193454 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:31.193890 3445 server_base.cc:1034] running on GCE node
I20260807 08:29:31.194207 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:31.194756 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:31.195922 3445 hybrid_clock.cc:648] HybridClock initialized: now 1786091371195896 us; error 34 us; skew 500 ppm
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:31.198448 3445 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:31.199184 3445 webserver.cc:460] Webserver started at http://127.1.157.65:44113/ using document root <none> and password file <none>
I20260807 08:29:31.199656 3445 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:31.199720 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:31.199903 3445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:31.201719 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "611ce7ba68c64984af55d6f064f343e0"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "1ef04fa6f287e00eb215cdc3a45e31c7"
server_key_iv: "7454dc959f0bf8f64c7673efc7632f6d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.202215 3445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "611ce7ba68c64984af55d6f064f343e0"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "1ef04fa6f287e00eb215cdc3a45e31c7"
server_key_iv: "7454dc959f0bf8f64c7673efc7632f6d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.205471 3445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:31.207674 3472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.208701 3445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:31.208810 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "611ce7ba68c64984af55d6f064f343e0"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "1ef04fa6f287e00eb215cdc3a45e31c7"
server_key_iv: "7454dc959f0bf8f64c7673efc7632f6d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.208900 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:31.237912 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:31.240363 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:31.240520 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:31.241876 3445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:31.241925 3445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.241976 3445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:31.241999 3445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:31.252140 3445 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:41961
I20260807 08:29:31.252266 3662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:41961 every 8 connection(s)
I20260807 08:29:31.252949 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:31.254510 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.242488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:41663 (local address 127.1.157.126:34005)
0807 08:29:31.242649 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.242654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.243956 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 08:29:31.244919 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.244922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.244982 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.245078 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.246141 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.246723 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.247793 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.248068 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.248168 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.250979 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.250983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.250986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.251013 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.252559 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.253048 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.253052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.253054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.253138 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.253511 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.253514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.253516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.253743 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.253842 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.254245 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.254382 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:29:31.255226 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.243389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.65:41663)
0807 08:29:31.243819 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.243867 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.244728 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.245232 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.245238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.245568 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.245998 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.246010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.246845 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.246848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.247672 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.247682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.248273 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.248278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.248518 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.249146 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.249164 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.250831 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.252664 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.252669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.252681 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.252954 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.253220 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.253223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.253225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.253397 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.253846 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.253851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.254050 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.254204 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.254479 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
I20260807 08:29:31.256366 3664 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34005
I20260807 08:29:31.256505 3664 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:31.256923 3664 heartbeater.cc:511] Master 127.1.157.126:34005 requested a full tablet report, sending...
I20260807 08:29:31.257001 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3445
I20260807 08:29:31.257118 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:31.257408 1653 external_mini_cluster.cc:1428] Setting key 34da658cd8adca24983fe7e98e741bed
I20260807 08:29:31.258414 3341 ts_manager.cc:194] Registered new tserver with Master: 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961)
I20260807 08:29:31.259557 3341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:41663
I20260807 08:29:31.265637 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.258464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:59573 (local address 127.1.157.126:34005)
0807 08:29:31.258562 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.258566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.258784 (+ 218us) server_negotiation.cc:407] Connection header received
0807 08:29:31.258896 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.258899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.258938 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.258999 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.259544 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.260230 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.261318 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.261640 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.261748 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.262860 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.262863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.262866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.262889 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.264153 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.264540 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.264543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.264545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.264631 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.264877 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.264880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.264882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.265077 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.265172 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.265409 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.265513 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:31.265635 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.258418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.65:59573)
0807 08:29:31.258706 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.258719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.258822 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.259099 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.259102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.259299 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.259419 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.259426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.260441 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.260444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.261202 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.261212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.261775 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.261778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.261867 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.262282 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.262296 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.262704 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.264255 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.264259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.264260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.264435 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.264721 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.264723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.264724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.264801 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.265154 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.265155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.265229 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.265325 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.265452 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:31.283391 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34005
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:31.388149 3675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:31.388372 3675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:31.388430 3675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:31.392246 3675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:31.392292 3675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:31.392342 3675 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:31.392395 3675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:31.397202 3675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:31.398010 3675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:31.398888 3675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:31.407235 3688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:31.407250 3682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:31.407347 3675 server_base.cc:1034] running on GCE node
W20260807 08:29:31.407244 3683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:31.407855 3675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:31.408390 3675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:31.409552 3675 hybrid_clock.cc:648] HybridClock initialized: now 1786091371409533 us; error 27 us; skew 500 ppm
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:31.412115 3675 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:31.412992 3675 webserver.cc:460] Webserver started at http://127.1.157.66:42301/ using document root <none> and password file <none>
I20260807 08:29:31.413496 3675 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:31.413560 3675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:31.413755 3675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:31.415686 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "ffcf5e2303944b54820aa14f647c9f72"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "33177d4c21a4c64db5ab5b9d1b0c0b16"
server_key_iv: "979ebe7baab1cac4afab51093fd15b82"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.416195 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "ffcf5e2303944b54820aa14f647c9f72"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "33177d4c21a4c64db5ab5b9d1b0c0b16"
server_key_iv: "979ebe7baab1cac4afab51093fd15b82"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.419740 3675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:31.421850 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.422937 3675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:31.423038 3675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "ffcf5e2303944b54820aa14f647c9f72"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "33177d4c21a4c64db5ab5b9d1b0c0b16"
server_key_iv: "979ebe7baab1cac4afab51093fd15b82"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.423127 3675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:31.446197 3675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:31.448632 3675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:31.448787 3675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:31.450040 3675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:31.450088 3675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.450138 3675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:31.450167 3675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:31.460300 3675 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:36085
I20260807 08:29:31.460397 3885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:36085 every 8 connection(s)
I20260807 08:29:31.461313 3675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:31.463479 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.450729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40871 (local address 127.1.157.126:34005)
0807 08:29:31.450837 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.450841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.451761 (+ 920us) server_negotiation.cc:407] Connection header received
0807 08:29:31.452648 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.452651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.452719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.452796 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.454170 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.454781 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.455976 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.456313 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.456426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.459892 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.459896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.459898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.459929 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.461505 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.462065 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.462069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.462070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.462168 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.462575 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.462578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.462579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.462773 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.462855 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.463272 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.463366 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:29:31.464191 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.451194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.66:40871)
0807 08:29:31.451623 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.451661 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.452451 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.453090 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.453099 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.453509 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.454025 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.454044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.454906 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.454934 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.455856 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.455866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.456492 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.456497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.456725 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.457310 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.457331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.459740 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.461630 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.461635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.461646 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.461972 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.462286 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.462288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.462290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.462477 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.462861 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.462865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.463064 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.463232 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.463451 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260807 08:29:31.465216 3888 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34005
I20260807 08:29:31.465356 3888 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:31.465794 3888 heartbeater.cc:511] Master 127.1.157.126:34005 requested a full tablet report, sending...
I20260807 08:29:31.466741 3339 ts_manager.cc:194] Registered new tserver with Master: ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085)
I20260807 08:29:31.467334 3339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:40871
I20260807 08:29:31.468631 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3675
I20260807 08:29:31.468743 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:31.469004 1653 external_mini_cluster.cc:1428] Setting key 193d57660b8eec679f8171b73126213c
I20260807 08:29:31.475543 3897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.467112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.66:58153)
0807 08:29:31.467410 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.467421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.467512 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.467735 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.467738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.467922 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.468032 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.468038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.468722 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.468725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.469356 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.469365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.469818 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.469821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.469893 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.472151 (+ 2258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.472163 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.472469 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.474148 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.474150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.474152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.474325 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.474612 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.474614 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.474615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.474685 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.475165 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.475168 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.475250 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.475330 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.475424 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260807 08:29:31.475653 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.467174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58153 (local address 127.1.157.126:34005)
0807 08:29:31.467288 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.467292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.467490 (+ 198us) server_negotiation.cc:407] Connection header received
0807 08:29:31.467601 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.467603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.467636 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.467697 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.468136 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.468603 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.469462 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.469713 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.469813 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.472592 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.472595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.472598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.472624 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.474048 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.474440 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.474444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.474446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.474530 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.474781 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.474784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.474786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.475050 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.475151 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.475407 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.475509 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:31.493371 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34005
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:31.597218 3900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:31.597432 3900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:31.597487 3900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:31.601344 3900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:31.601394 3900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:31.601444 3900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:31.601497 3900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:31.606776 3900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:31.607630 3900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:31.608520 3900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:31.617370 3906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:31.617362 3909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:31.617434 3912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:31.617607 3900 server_base.cc:1034] running on GCE node
I20260807 08:29:31.617972 3900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:31.618554 3900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:31.619707 3900 hybrid_clock.cc:648] HybridClock initialized: now 1786091371619696 us; error 27 us; skew 500 ppm
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:31.622123 3900 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:31.622920 3900 webserver.cc:460] Webserver started at http://127.1.157.67:41197/ using document root <none> and password file <none>
I20260807 08:29:31.623394 3900 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:31.623457 3900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:31.623627 3900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:31.625545 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "5f4e3679445345678f97ceae2fb22207"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "4e0b082520dab864f2e2700cf45e55ef"
server_key_iv: "057042c90a1535b1617166a9b0b3444d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.626096 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "5f4e3679445345678f97ceae2fb22207"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "4e0b082520dab864f2e2700cf45e55ef"
server_key_iv: "057042c90a1535b1617166a9b0b3444d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.629429 3900 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:31.631574 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.632608 3900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:31.632710 3900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "5f4e3679445345678f97ceae2fb22207"
format_stamp: "Formatted at 2026-08-07 08:29:31 on dist-test-slave-6zbq"
server_key: "4e0b082520dab864f2e2700cf45e55ef"
server_key_iv: "057042c90a1535b1617166a9b0b3444d"
server_key_version: "encryptionkey@0"
I20260807 08:29:31.632802 3900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:31.653911 3900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:31.656255 3900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:31.656404 3900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:31.657672 3900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:31.657718 3900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:31.657776 3900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:31.657801 3900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:31.668319 3900 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:41289
I20260807 08:29:31.668386 4094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:41289 every 8 connection(s)
I20260807 08:29:31.669219 3900 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:31.671295 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.658314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:40821 (local address 127.1.157.126:34005)
0807 08:29:31.658439 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.658443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.659977 (+ 1534us) server_negotiation.cc:407] Connection header received
0807 08:29:31.660996 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.660999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.661058 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.661161 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.662608 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.663415 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.665025 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.665361 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.665468 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.668093 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.668096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.668100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.668154 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.669548 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.670093 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.670096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.670097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.670178 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.670474 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.670476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.670477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.670637 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.670710 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.671080 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.671172 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:29:31.671959 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.659342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.67:40821)
0807 08:29:31.659834 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.659885 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.660788 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.661342 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.661350 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.661777 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.662454 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.662474 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.663550 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.663553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.664898 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.664912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.665542 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.665546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.665746 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.666284 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.666305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.667916 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.669672 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.669677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.669690 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.669996 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.670241 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.670243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.670244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.670393 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.670711 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.670715 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.670903 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.671034 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.671253 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260807 08:29:31.673085 4095 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34005
I20260807 08:29:31.673223 4095 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:31.673683 4095 heartbeater.cc:511] Master 127.1.157.126:34005 requested a full tablet report, sending...
I20260807 08:29:31.674610 3341 ts_manager.cc:194] Registered new tserver with Master: 5f4e3679445345678f97ceae2fb22207 (127.1.157.67:41289)
I20260807 08:29:31.675202 3341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:40821
I20260807 08:29:31.678476 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3900
I20260807 08:29:31.678583 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:31.678841 1653 external_mini_cluster.cc:1428] Setting key 6421220f0af0924ed8c85a26de747fc5
I20260807 08:29:31.680727 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:29:31.681348 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.674962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.1.157.67:36695)
0807 08:29:31.675198 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.675209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.675274 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.675667 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.675669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.675818 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:31.675909 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.675913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.676828 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.676830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.677352 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.677359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.677731 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.677733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.677792 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.678134 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:31.678144 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:31.678475 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:31.680067 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.680070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.680072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.680251 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.680509 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:31.680511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:31.680513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:31.680600 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:31.680976 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:31.680978 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:31.681049 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.681138 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.681232 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
I20260807 08:29:31.681449 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.675052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:36695 (local address 127.1.157.126:34005)
0807 08:29:31.675476 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.675480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.675492 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:31.675521 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.675524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.675560 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.675642 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.676016 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.676715 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.677453 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.677653 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.677726 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.678592 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.678595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.678597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.678618 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.679975 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.680334 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.680338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.680340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.680428 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.680686 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.680689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.680690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.680895 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.680983 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.681209 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.681315 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364}
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:29:31 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:31.704589 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.697742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36140 (local address 127.1.157.126:34005)
0807 08:29:31.697836 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.697839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.697850 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:29:31.697878 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.697880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.697911 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.697970 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:31.698595 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.699051 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.699766 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.699950 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.700010 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.702056 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:31.702058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:31.702060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:31.702081 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:31.703156 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.703523 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.703527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.703529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.703599 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:31.703844 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:31.703848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:31.703849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:31.704004 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:31.704118 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.704350 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.704459 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:29:31.706980 3341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:31.709772 3341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:31.723348 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.717997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.1.157.67:41289)
0807 08:29:31.718333 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.718500 (+ 167us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.718528 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:29:31.718602 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.718607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.718804 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.718956 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:31.719375 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720443 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.721876 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.722687 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.722795 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.722947 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.723121 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.723215 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":102,"threads_started":1}
I20260807 08:29:31.723340 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.717388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:41289 (local address 127.0.0.1:43060)
0807 08:29:31.717674 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.717717 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.717830 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.719107 (+ 1277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.719110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.719127 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:31.719236 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.719243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720552 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.720555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.721744 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.721755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.722811 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.722814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.722936 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.722975 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.723091 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.723206 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":33,"thread_start_us":69,"threads_started":1}
I20260807 08:29:31.723639 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.717664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.1.157.66:36085)
0807 08:29:31.718545 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.718556 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.718579 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:29:31.718652 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.718658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.718819 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.719289 (+ 470us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:31.719427 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720261 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.721945 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.722725 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.722809 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.722947 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.723413 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.723512 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":107,"threads_started":1}
I20260807 08:29:31.723663 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.718480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.1.157.65:41961)
0807 08:29:31.719355 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.719369 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.719389 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:29:31.719459 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.719464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.719646 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.719795 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:31.720008 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720876 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.722093 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.722699 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.722793 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.722946 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.723415 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.723515 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":69,"threads_started":1}
I20260807 08:29:31.723676 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.717494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36085 (local address 127.0.0.1:56202)
0807 08:29:31.718014 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.718023 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.718094 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.718983 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.718988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.719011 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:31.719127 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.719133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720353 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.720357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.721826 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.721834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.723012 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.723016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.723126 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.723166 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.723232 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.723299 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":27,"thread_start_us":106,"threads_started":1}
I20260807 08:29:31.723837 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.718298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41961 (local address 127.0.0.1:37858)
0807 08:29:31.718993 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.719006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.719076 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.719794 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.719797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.719815 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:31.719899 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.719905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.720978 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.720981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.721953 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.721963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.723007 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.723010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.723157 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.723179 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.723242 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.723305 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":46,"thread_start_us":549,"threads_started":1}
I20260807 08:29:31.726179 3996 tablet_service.cc:1467] Processing CreateTablet for tablet c612ba045ae74fba99706e3dd693b2e0 (DEFAULT_TABLE table=test-table [id=ec74de4129de45d594fe190aa260350d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:31.726468 3558 tablet_service.cc:1467] Processing CreateTablet for tablet c612ba045ae74fba99706e3dd693b2e0 (DEFAULT_TABLE table=test-table [id=ec74de4129de45d594fe190aa260350d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:31.726605 3779 tablet_service.cc:1467] Processing CreateTablet for tablet c612ba045ae74fba99706e3dd693b2e0 (DEFAULT_TABLE table=test-table [id=ec74de4129de45d594fe190aa260350d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:31.727264 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c612ba045ae74fba99706e3dd693b2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:31.727463 3558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c612ba045ae74fba99706e3dd693b2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:31.727463 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c612ba045ae74fba99706e3dd693b2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:31.733361 4122 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Bootstrap starting.
I20260807 08:29:31.735519 4124 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Bootstrap starting.
I20260807 08:29:31.735555 4122 tablet_bootstrap.cc:654] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:31.736297 4122 log.cc:824] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:31.736845 4123 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Bootstrap starting.
I20260807 08:29:31.737330 4124 tablet_bootstrap.cc:654] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:31.738114 4124 log.cc:824] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:31.738229 4122 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: No bootstrap required, opened a new log
I20260807 08:29:31.738467 4122 ts_tablet_manager.cc:1397] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:29:31.738552 4123 tablet_bootstrap.cc:654] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:31.739248 4123 log.cc:824] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:31.740031 4124 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: No bootstrap required, opened a new log
I20260807 08:29:31.740268 4124 ts_tablet_manager.cc:1397] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:29:31.740942 4123 tablet_bootstrap.cc:492] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: No bootstrap required, opened a new log
I20260807 08:29:31.741114 4123 ts_tablet_manager.cc:1397] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:29:31.742355 4122 raft_consensus.cc:348] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.742602 4122 raft_consensus.cc:374] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:31.742692 4122 raft_consensus.cc:729] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4e3679445345678f97ceae2fb22207, State: Initialized, Role: FOLLOWER
I20260807 08:29:31.743127 4122 consensus_queue.cc:260] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.743825 4095 heartbeater.cc:503] Master 127.1.157.126:34005 was elected leader, sending a full tablet report...
I20260807 08:29:31.744086 4122 ts_tablet_manager.cc:1428] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:29:31.744103 4123 raft_consensus.cc:348] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.744334 4123 raft_consensus.cc:374] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:31.744380 4123 raft_consensus.cc:729] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcf5e2303944b54820aa14f647c9f72, State: Initialized, Role: FOLLOWER
I20260807 08:29:31.744639 4124 raft_consensus.cc:348] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.744773 4123 consensus_queue.cc:260] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.744920 4124 raft_consensus.cc:374] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:31.744983 4124 raft_consensus.cc:729] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ce7ba68c64984af55d6f064f343e0, State: Initialized, Role: FOLLOWER
I20260807 08:29:31.745441 3888 heartbeater.cc:503] Master 127.1.157.126:34005 was elected leader, sending a full tablet report...
I20260807 08:29:31.745765 4123 ts_tablet_manager.cc:1428] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:29:31.746186 4124 consensus_queue.cc:260] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.746766 3664 heartbeater.cc:503] Master 127.1.157.126:34005 was elected leader, sending a full tablet report...
I20260807 08:29:31.747057 4124 ts_tablet_manager.cc:1428] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260807 08:29:31.753952 3665 tablet.cc:2369] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:31.920120 4096 tablet.cc:2369] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:31.962684 3889 tablet.cc:2369] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:31.968626 4130 raft_consensus.cc:482] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:31.968787 4130 raft_consensus.cc:504] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.969864 4130 leader_election.cc:288] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085), 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961)
I20260807 08:29:31.973922 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.970103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41961 (local address 127.1.157.67:36147)
0807 08:29:31.970477 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.970490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.970573 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.970831 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.970837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.970860 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:31.971014 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.971021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.971770 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.971772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.972840 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.972851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.973593 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.973596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.973681 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.973703 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.973760 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.973811 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260807 08:29:31.973984 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.970296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:36147 (local address 127.1.157.65:41961)
0807 08:29:31.970446 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.970450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.970560 (+ 110us) server_negotiation.cc:407] Connection header received
0807 08:29:31.970674 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.970679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.970727 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.970798 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:31.971128 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.971661 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.972960 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.973483 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.973598 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.973654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.973783 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.973856 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 08:29:31.974613 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c612ba045ae74fba99706e3dd693b2e0" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ce7ba68c64984af55d6f064f343e0" is_pre_election: true
I20260807 08:29:31.974674 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.970139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:35437 (local address 127.1.157.66:36085)
0807 08:29:31.970246 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:31.970262 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:31.970475 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:29:31.970528 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:31.970530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:31.970569 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:31.970632 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:31.971753 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.972329 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.973556 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.974088 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.974209 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.974258 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:31.974505 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 08:29:31.974577 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:29:31.974674 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:31.970057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36085 (local address 127.1.157.67:35437)
0807 08:29:31.970243 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:31.970256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:31.970379 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:31.970958 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:31.970960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:31.970974 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:31.971630 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.971641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.972454 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.972458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.973445 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:31.973456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:31.974194 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:31.974197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:31.974366 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:31.974393 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:29:31.974461 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:29:31.974543 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":3}
I20260807 08:29:31.974905 3591 raft_consensus.cc:2454] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 0.
I20260807 08:29:31.975358 3930 leader_election.cc:302] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e3679445345678f97ceae2fb22207, 611ce7ba68c64984af55d6f064f343e0; no voters:
I20260807 08:29:31.975517 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c612ba045ae74fba99706e3dd693b2e0" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72" is_pre_election: true
I20260807 08:29:31.975596 4130 raft_consensus.cc:2781] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:31.975674 4130 raft_consensus.cc:482] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:31.975724 4130 raft_consensus.cc:3037] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:31.975775 3808 raft_consensus.cc:2454] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 0.
I20260807 08:29:31.976711 4130 raft_consensus.cc:504] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.976990 4130 leader_election.cc:288] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 election: Requested vote from peers ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085), 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961)
I20260807 08:29:31.977332 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c612ba045ae74fba99706e3dd693b2e0" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72"
I20260807 08:29:31.977345 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c612ba045ae74fba99706e3dd693b2e0" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ce7ba68c64984af55d6f064f343e0"
I20260807 08:29:31.977456 3808 raft_consensus.cc:3037] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:31.977456 3591 raft_consensus.cc:3037] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:31.978718 3591 raft_consensus.cc:2454] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 1.
I20260807 08:29:31.978776 3808 raft_consensus.cc:2454] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 1.
I20260807 08:29:31.979007 3930 leader_election.cc:302] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e3679445345678f97ceae2fb22207, 611ce7ba68c64984af55d6f064f343e0; no voters:
I20260807 08:29:31.979197 4130 raft_consensus.cc:2781] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:31.979378 4130 raft_consensus.cc:686] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4e3679445345678f97ceae2fb22207, State: Running, Role: LEADER
I20260807 08:29:31.979665 4130 consensus_queue.cc:237] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } }
I20260807 08:29:31.982414 3337 catalog_manager.cc:5495] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f4e3679445345678f97ceae2fb22207 (127.1.157.67). New cstate: current_term: 1 leader_uuid: "5f4e3679445345678f97ceae2fb22207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:32.017853 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.014641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.1.157.67:41289)
0807 08:29:32.014785 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.014790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.014865 (+ 75us) server_negotiation.cc:407] Connection header received
0807 08:29:32.015024 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.015026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.015093 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.015173 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:32.015376 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.015844 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.016629 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.016840 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.016912 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.017148 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.017531 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.017616 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.017674 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.017720 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:29:32.023408 3808 raft_consensus.cc:1261] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Refusing update from remote peer 5f4e3679445345678f97ceae2fb22207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.024050 4130 consensus_queue.cc:1035] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.025738 3591 raft_consensus.cc:1261] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Refusing update from remote peer 5f4e3679445345678f97ceae2fb22207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.026252 4130 consensus_queue.cc:1035] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.031838 4140 mvcc.cc:205] Tried to move back new op lower bound from 7315830259800440832 to 7315830259630686208. Current Snapshot: MvccSnapshot[applied={T|T < 7315830259800440832 or (T in {7315830259800440832})}]
I20260807 08:29:32.042315 3341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:32.042575 3341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:32.045060 3341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Sending DeleteTablet for 3 replicas of tablet c612ba045ae74fba99706e3dd693b2e0
I20260807 08:29:32.045778 3996 tablet_service.cc:1514] Processing DeleteTablet for tablet c612ba045ae74fba99706e3dd693b2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:43060
I20260807 08:29:32.046087 3558 tablet_service.cc:1514] Processing DeleteTablet for tablet c612ba045ae74fba99706e3dd693b2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37858
I20260807 08:29:32.046310 4152 tablet_replica.cc:330] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: stopping tablet replica
I20260807 08:29:32.046299 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet c612ba045ae74fba99706e3dd693b2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:56202
I20260807 08:29:32.046617 4152 raft_consensus.cc:2229] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:32.046811 4153 tablet_replica.cc:330] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: stopping tablet replica
I20260807 08:29:32.047050 4152 raft_consensus.cc:2258] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.047119 4153 raft_consensus.cc:2229] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:32.047432 4153 raft_consensus.cc:2258] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.047637 3341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:32.048007 3341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:32.048656 4152 ts_tablet_manager.cc:1905] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.048771 4153 ts_tablet_manager.cc:1905] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.051824 4153 ts_tablet_manager.cc:1918] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:32.051966 4153 log.cc:1196] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/c612ba045ae74fba99706e3dd693b2e0
I20260807 08:29:32.052313 4153 ts_tablet_manager.cc:1939] T c612ba045ae74fba99706e3dd693b2e0 P 611ce7ba68c64984af55d6f064f343e0: Deleting consensus metadata
I20260807 08:29:32.052424 4154 tablet_replica.cc:330] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: stopping tablet replica
I20260807 08:29:32.052764 4154 raft_consensus.cc:2229] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:32.052986 4152 ts_tablet_manager.cc:1918] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:32.053122 4152 log.cc:1196] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/c612ba045ae74fba99706e3dd693b2e0
I20260807 08:29:32.053550 3558 tablet_service.cc:1467] Processing CreateTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 (DEFAULT_TABLE table=test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:32.053797 3558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b91195197be4e97be4c56cfc07cf1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.053663 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 (DEFAULT_TABLE table=test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:32.054054 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b91195197be4e97be4c56cfc07cf1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.055961 4124 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Bootstrap starting.
I20260807 08:29:32.056833 4123 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Bootstrap starting.
I20260807 08:29:32.057224 4124 tablet_bootstrap.cc:654] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.058077 4123 tablet_bootstrap.cc:654] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.058624 3314 catalog_manager.cc:4841] TS 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961): tablet c612ba045ae74fba99706e3dd693b2e0 (table test-table [id=ec74de4129de45d594fe190aa260350d]) successfully deleted
I20260807 08:29:32.053720 3996 tablet_service.cc:1467] Processing CreateTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 (DEFAULT_TABLE table=test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:32.059319 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b91195197be4e97be4c56cfc07cf1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.059432 4152 ts_tablet_manager.cc:1939] T c612ba045ae74fba99706e3dd693b2e0 P 5f4e3679445345678f97ceae2fb22207: Deleting consensus metadata
I20260807 08:29:32.060212 4123 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: No bootstrap required, opened a new log
I20260807 08:29:32.060282 4123 ts_tablet_manager.cc:1397] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:29:32.060412 4154 raft_consensus.cc:2258] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.060891 3313 catalog_manager.cc:4841] TS 5f4e3679445345678f97ceae2fb22207 (127.1.157.67:41289): tablet c612ba045ae74fba99706e3dd693b2e0 (table test-table [id=ec74de4129de45d594fe190aa260350d]) successfully deleted
I20260807 08:29:32.060907 4123 raft_consensus.cc:348] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.061009 4123 raft_consensus.cc:374] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.061038 4123 raft_consensus.cc:729] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcf5e2303944b54820aa14f647c9f72, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.061163 4123 consensus_queue.cc:260] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.061406 4123 ts_tablet_manager.cc:1428] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:29:32.061522 4122 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Bootstrap starting.
I20260807 08:29:32.061654 4124 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: No bootstrap required, opened a new log
I20260807 08:29:32.061721 4124 ts_tablet_manager.cc:1397] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 08:29:32.062139 4124 raft_consensus.cc:348] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.062282 4124 raft_consensus.cc:374] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.062379 4124 raft_consensus.cc:729] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ce7ba68c64984af55d6f064f343e0, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.062427 4122 tablet_bootstrap.cc:654] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.062597 4124 consensus_queue.cc:260] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.062959 4124 ts_tablet_manager.cc:1428] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:32.063513 4154 ts_tablet_manager.cc:1905] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.064191 4122 tablet_bootstrap.cc:492] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: No bootstrap required, opened a new log
I20260807 08:29:32.064270 4122 ts_tablet_manager.cc:1397] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 08:29:32.064868 4122 raft_consensus.cc:348] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.064967 4122 raft_consensus.cc:374] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.065001 4122 raft_consensus.cc:729] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4e3679445345678f97ceae2fb22207, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.065135 4122 consensus_queue.cc:260] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.065450 4122 ts_tablet_manager.cc:1428] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:29:32.068246 4154 ts_tablet_manager.cc:1918] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:32.068334 4154 log.cc:1196] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/c612ba045ae74fba99706e3dd693b2e0
I20260807 08:29:32.068629 4154 ts_tablet_manager.cc:1939] T c612ba045ae74fba99706e3dd693b2e0 P ffcf5e2303944b54820aa14f647c9f72: Deleting consensus metadata
I20260807 08:29:32.069628 3312 catalog_manager.cc:4841] TS ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085): tablet c612ba045ae74fba99706e3dd693b2e0 (table test-table [id=ec74de4129de45d594fe190aa260350d]) successfully deleted
I20260807 08:29:32.118469 4130 raft_consensus.cc:482] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:32.118579 4130 raft_consensus.cc:504] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.118933 4130 leader_election.cc:288] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961), ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085)
I20260807 08:29:32.119375 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b91195197be4e97be4c56cfc07cf1a6" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ce7ba68c64984af55d6f064f343e0" is_pre_election: true
I20260807 08:29:32.119410 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b91195197be4e97be4c56cfc07cf1a6" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72" is_pre_election: true
I20260807 08:29:32.119529 3591 raft_consensus.cc:2454] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 0.
I20260807 08:29:32.119570 3808 raft_consensus.cc:2454] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 0.
I20260807 08:29:32.119823 3930 leader_election.cc:302] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e3679445345678f97ceae2fb22207, 611ce7ba68c64984af55d6f064f343e0; no voters:
I20260807 08:29:32.119989 4130 raft_consensus.cc:2781] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:32.120060 4130 raft_consensus.cc:482] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:32.120091 4130 raft_consensus.cc:3037] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.120981 4130 raft_consensus.cc:504] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.121289 4130 leader_election.cc:288] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 election: Requested vote from peers 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961), ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085)
I20260807 08:29:32.121587 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b91195197be4e97be4c56cfc07cf1a6" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72"
I20260807 08:29:32.121593 3591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b91195197be4e97be4c56cfc07cf1a6" candidate_uuid: "5f4e3679445345678f97ceae2fb22207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ce7ba68c64984af55d6f064f343e0"
I20260807 08:29:32.121704 3808 raft_consensus.cc:3037] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.121706 3591 raft_consensus.cc:3037] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.122735 3808 raft_consensus.cc:2454] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 1.
I20260807 08:29:32.123003 3926 leader_election.cc:302] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e3679445345678f97ceae2fb22207, ffcf5e2303944b54820aa14f647c9f72; no voters:
I20260807 08:29:32.123055 3591 raft_consensus.cc:2454] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4e3679445345678f97ceae2fb22207 in term 1.
I20260807 08:29:32.123163 4130 raft_consensus.cc:2781] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:32.123242 4130 raft_consensus.cc:686] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4e3679445345678f97ceae2fb22207, State: Running, Role: LEADER
I20260807 08:29:32.123370 4130 consensus_queue.cc:237] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.124831 3337 catalog_manager.cc:5495] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f4e3679445345678f97ceae2fb22207 (127.1.157.67). New cstate: current_term: 1 leader_uuid: "5f4e3679445345678f97ceae2fb22207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } health_report { overall_health: HEALTHY } } }
Aug 07 08:29:32 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091370, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:32.160729 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.151890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.1.157.126:34005)
0807 08:29:32.152016 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.152021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.152238 (+ 217us) server_negotiation.cc:407] Connection header received
0807 08:29:32.152378 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.152380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.152417 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.152479 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:32.153387 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.153890 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.155020 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.155226 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.155291 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.157833 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:32.157835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:32.157837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:32.157857 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:32.159207 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.159604 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.159607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.159609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.159682 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.159911 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.159914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.159916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.160087 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:32.160168 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.160442 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.160577 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":47}
I20260807 08:29:32.160729 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.151781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34005 (local address 127.0.0.1:36152)
0807 08:29:32.152150 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.152166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.152287 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.152534 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.152537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.153110 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:32.153285 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.153295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.153990 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.153995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.154914 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.154927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.155320 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.155324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.155418 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.156010 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:32.156026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:32.157721 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:32.159298 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:32.159307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:32.159313 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:32.159520 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:32.159756 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:32.159760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:32.159761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:32.159839 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:32.160179 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:32.160186 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:32.160290 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.160383 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.160525 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":87,"threads_started":1}
I20260807 08:29:32.164548 3337 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36152:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:29:32.165239 3337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:32.169924 3779 tablet_service.cc:1467] Processing CreateTablet for tablet a16e973a94574173a29e0e0d57dd2456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44d759eb9bf941aa8961fcc057578a83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:32.170315 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16e973a94574173a29e0e0d57dd2456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.170288 3996 tablet_service.cc:1467] Processing CreateTablet for tablet a16e973a94574173a29e0e0d57dd2456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44d759eb9bf941aa8961fcc057578a83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:32.171353 3996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16e973a94574173a29e0e0d57dd2456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.169898 3558 tablet_service.cc:1467] Processing CreateTablet for tablet a16e973a94574173a29e0e0d57dd2456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=44d759eb9bf941aa8961fcc057578a83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:32.171991 3558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a16e973a94574173a29e0e0d57dd2456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:32.174500 4123 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: Bootstrap starting.
I20260807 08:29:32.174606 4122 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: Bootstrap starting.
I20260807 08:29:32.174911 4124 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: Bootstrap starting.
I20260807 08:29:32.175455 4123 tablet_bootstrap.cc:654] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.176330 4122 tablet_bootstrap.cc:654] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.176704 4124 tablet_bootstrap.cc:654] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:32.178292 4122 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: No bootstrap required, opened a new log
I20260807 08:29:32.178422 4122 ts_tablet_manager.cc:1397] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260807 08:29:32.178690 4124 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: No bootstrap required, opened a new log
I20260807 08:29:32.178776 4124 ts_tablet_manager.cc:1397] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:29:32.179126 4124 raft_consensus.cc:348] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.179224 4124 raft_consensus.cc:374] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.179255 4124 raft_consensus.cc:729] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ce7ba68c64984af55d6f064f343e0, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.179198 4122 raft_consensus.cc:348] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.179308 4122 raft_consensus.cc:374] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.179356 4122 raft_consensus.cc:729] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4e3679445345678f97ceae2fb22207, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.179369 4124 consensus_queue.cc:260] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.179486 4122 consensus_queue.cc:260] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.179610 4124 ts_tablet_manager.cc:1428] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:32.179785 4130 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.179915 4130 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.179940 4134 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.180058 4134 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.180577 4122 ts_tablet_manager.cc:1428] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:29:32.181540 4123 tablet_bootstrap.cc:492] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: No bootstrap required, opened a new log
I20260807 08:29:32.181620 4123 ts_tablet_manager.cc:1397] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260807 08:29:32.182243 4123 raft_consensus.cc:348] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.182374 4123 raft_consensus.cc:374] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:32.182411 4123 raft_consensus.cc:729] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcf5e2303944b54820aa14f647c9f72, State: Initialized, Role: FOLLOWER
I20260807 08:29:32.182520 4123 consensus_queue.cc:260] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.182765 4132 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.182968 4132 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.182794 4123 ts_tablet_manager.cc:1428] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:32.457643 4134 raft_consensus.cc:482] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:32.457784 4134 raft_consensus.cc:504] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.459071 4134 leader_election.cc:288] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085), 5f4e3679445345678f97ceae2fb22207 (127.1.157.67:41289)
I20260807 08:29:32.463230 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.459405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:47209 (local address 127.1.157.67:41289)
0807 08:29:32.459626 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.459631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.459903 (+ 272us) server_negotiation.cc:407] Connection header received
0807 08:29:32.460013 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.460016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.460066 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.460168 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:32.460514 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.461237 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.462278 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.462737 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.462842 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.462900 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.463046 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.463118 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260807 08:29:32.463243 4170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.459301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:41289 (local address 127.1.157.65:47209)
0807 08:29:32.459777 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.459789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.459922 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.460283 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.460286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.460305 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:32.460395 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.460401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.461359 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.461363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.462159 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.462168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.462842 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.462846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.462929 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.462954 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.463021 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.463086 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":96,"threads_started":1}
I20260807 08:29:32.463757 4021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16e973a94574173a29e0e0d57dd2456" candidate_uuid: "611ce7ba68c64984af55d6f064f343e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f4e3679445345678f97ceae2fb22207" is_pre_election: true
I20260807 08:29:32.463989 4021 raft_consensus.cc:2454] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ce7ba68c64984af55d6f064f343e0 in term 0.
I20260807 08:29:32.464402 3476 leader_election.cc:302] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f4e3679445345678f97ceae2fb22207, 611ce7ba68c64984af55d6f064f343e0; no voters:
I20260807 08:29:32.464573 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.459369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:43829 (local address 127.1.157.66:36085)
0807 08:29:32.459560 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.459565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.459746 (+ 181us) server_negotiation.cc:407] Connection header received
0807 08:29:32.459877 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.459880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.459933 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.460004 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:32.460332 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.461104 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.463146 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.463729 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.463813 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.463857 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.464380 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.464467 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 08:29:32.464601 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.459262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36085 (local address 127.1.157.65:43829)
0807 08:29:32.459625 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.459639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.459776 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.460019 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.460022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.460060 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:32.460190 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.460198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.461223 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.461226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.462988 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.463000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.464133 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.464136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.464228 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.464250 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.464352 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.464419 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":124,"threads_started":1}
I20260807 08:29:32.464725 4134 raft_consensus.cc:2781] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:32.464778 4134 raft_consensus.cc:482] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:32.464812 4134 raft_consensus.cc:3037] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.465198 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16e973a94574173a29e0e0d57dd2456" candidate_uuid: "611ce7ba68c64984af55d6f064f343e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72" is_pre_election: true
I20260807 08:29:32.465348 3808 raft_consensus.cc:2454] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ce7ba68c64984af55d6f064f343e0 in term 0.
I20260807 08:29:32.466080 4134 raft_consensus.cc:504] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.466411 4134 leader_election.cc:288] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [CANDIDATE]: Term 1 election: Requested vote from peers ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085), 5f4e3679445345678f97ceae2fb22207 (127.1.157.67:41289)
I20260807 08:29:32.466706 3808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16e973a94574173a29e0e0d57dd2456" candidate_uuid: "611ce7ba68c64984af55d6f064f343e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffcf5e2303944b54820aa14f647c9f72"
I20260807 08:29:32.466820 3808 raft_consensus.cc:3037] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.466789 4021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a16e973a94574173a29e0e0d57dd2456" candidate_uuid: "611ce7ba68c64984af55d6f064f343e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f4e3679445345678f97ceae2fb22207"
I20260807 08:29:32.466897 4021 raft_consensus.cc:3037] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:32.467829 3808 raft_consensus.cc:2454] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ce7ba68c64984af55d6f064f343e0 in term 1.
I20260807 08:29:32.468080 4021 raft_consensus.cc:2454] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ce7ba68c64984af55d6f064f343e0 in term 1.
I20260807 08:29:32.468652 3475 leader_election.cc:302] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ce7ba68c64984af55d6f064f343e0, ffcf5e2303944b54820aa14f647c9f72; no voters:
I20260807 08:29:32.468823 4134 raft_consensus.cc:2781] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:32.469034 4134 raft_consensus.cc:686] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [term 1 LEADER]: Becoming Leader. State: Replica: 611ce7ba68c64984af55d6f064f343e0, State: Running, Role: LEADER
I20260807 08:29:32.469281 4134 consensus_queue.cc:237] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } }
I20260807 08:29:32.470818 4172 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: TxnStatusTablet state changed. Reason: New leader 611ce7ba68c64984af55d6f064f343e0. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.470927 4172 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:32.471500 4174 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:29:32.471683 4174 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:29:32.471895 3337 catalog_manager.cc:5495] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:32.544901 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.540414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41961 (local address 127.0.0.1:37874)
0807 08:29:32.540571 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.540588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.540692 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.541017 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.541019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.541024 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:32.541133 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.541137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.542502 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.542504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.543096 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.543103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.543793 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.543796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.543859 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.543901 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.544681 (+ 780us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.544691 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.544726 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.544794 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:29:32.545120 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.540477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.1.157.65:41961)
0807 08:29:32.540740 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.540744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.540760 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:29:32.540838 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.540841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.540904 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.541025 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:32.541226 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.542406 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.543199 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.543693 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.543804 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.544029 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.544601 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.544714 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.544750 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.544805 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260807 08:29:32.548028 3808 raft_consensus.cc:1261] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Refusing update from remote peer 611ce7ba68c64984af55d6f064f343e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.548110 4021 raft_consensus.cc:1261] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Refusing update from remote peer 611ce7ba68c64984af55d6f064f343e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.548492 4172 consensus_queue.cc:1035] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.548646 4134 consensus_queue.cc:1035] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.550194 4132 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: TxnStatusTablet state changed. Reason: New leader 611ce7ba68c64984af55d6f064f343e0. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.550323 4132 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.550529 4130 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: TxnStatusTablet state changed. Reason: New leader 611ce7ba68c64984af55d6f064f343e0. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.550634 4130 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.552619 4172 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.552732 4172 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:32.552899 4132 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.552995 4132 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P ffcf5e2303944b54820aa14f647c9f72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.552989 4179 mvcc.cc:205] Tried to move back new op lower bound from 7315830261951873024 to 7315830261636325376. Current Snapshot: MvccSnapshot[applied={T|T < 7315830261951873024}]
I20260807 08:29:32.553678 4134 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.553785 4134 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 611ce7ba68c64984af55d6f064f343e0: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:32.555127 4130 tablet_replica.cc:439] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "611ce7ba68c64984af55d6f064f343e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 } } peers { permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 } } peers { permanent_uuid: "5f4e3679445345678f97ceae2fb22207" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41289 } } }
I20260807 08:29:32.555234 4130 tablet_replica.cc:442] T a16e973a94574173a29e0e0d57dd2456 P 5f4e3679445345678f97ceae2fb22207: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:32.554307 4181 mvcc.cc:205] Tried to move back new op lower bound from 7315830261951873024 to 7315830261636325376. Current Snapshot: MvccSnapshot[applied={T|T < 7315830261951873024}]
I20260807 08:29:32.570477 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.567150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:47445 (local address 127.1.157.65:41961)
0807 08:29:32.567284 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.567287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.567453 (+ 166us) server_negotiation.cc:407] Connection header received
0807 08:29:32.567569 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.567571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.567608 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.567666 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:32.567879 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.568337 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.569547 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.569805 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.569881 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.570136 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.570193 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.570238 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.570324 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.570388 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:29:32.570561 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.567069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41961 (local address 127.1.157.67:47445)
0807 08:29:32.567373 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.567389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.567493 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.567689 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.567692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.567699 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:32.567800 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.567806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.568427 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.568431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.569452 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.569463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.569897 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.569900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.569986 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.570035 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.570254 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.570259 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.570318 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.570423 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260807 08:29:32.577109 3591 raft_consensus.cc:1261] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Refusing update from remote peer 5f4e3679445345678f97ceae2fb22207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.577131 3808 raft_consensus.cc:1261] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Refusing update from remote peer 5f4e3679445345678f97ceae2fb22207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:32.577473 4136 consensus_queue.cc:1035] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ce7ba68c64984af55d6f064f343e0" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.577625 4130 consensus_queue.cc:1035] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffcf5e2303944b54820aa14f647c9f72" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:32.598018 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.593329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:53031 (local address 127.1.157.67:41289)
0807 08:29:32.593422 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.593426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.593668 (+ 242us) server_negotiation.cc:407] Connection header received
0807 08:29:32.593839 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.593843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.593883 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.593942 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:32.594319 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.594909 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.596644 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.596928 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.597027 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.597538 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.597623 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.597694 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.597847 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.597900 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 08:29:32.598453 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.593286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:41289 (local address 127.1.157.65:53031)
0807 08:29:32.593598 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:32.593611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:32.593746 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:32.593961 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:32.593963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:32.593970 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:32.594185 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.594194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.595087 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.595090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.596540 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:32.596552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.597228 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.597233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:32.597367 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.597411 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:32.597727 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:32.597734 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:29:32.597826 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 08:29:32.597900 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":157,"threads_started":1}
I20260807 08:29:32.626928 3341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:32.627077 3341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:32.628847 3341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcc4d42f8d62495c813dcdb60c409425: Sending DeleteTablet for 3 replicas of tablet 4b91195197be4e97be4c56cfc07cf1a6
I20260807 08:29:32.629383 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:56202
I20260807 08:29:32.629556 3996 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:43060
I20260807 08:29:32.629951 4217 tablet_replica.cc:330] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: stopping tablet replica
I20260807 08:29:32.629999 4210 tablet_replica.cc:330] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: stopping tablet replica
I20260807 08:29:32.630123 4217 raft_consensus.cc:2229] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:32.630262 4210 raft_consensus.cc:2229] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:32.630304 4217 raft_consensus.cc:2258] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.630481 4210 raft_consensus.cc:2258] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.630914 4217 ts_tablet_manager.cc:1905] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.631105 3558 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b91195197be4e97be4c56cfc07cf1a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:32 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37858
I20260807 08:29:32.631449 4210 ts_tablet_manager.cc:1905] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.631935 4219 tablet_replica.cc:330] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: stopping tablet replica
I20260807 08:29:32.632087 4219 raft_consensus.cc:2229] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:32.632275 4219 raft_consensus.cc:2258] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:32.633891 4217 ts_tablet_manager.cc:1918] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:32.633973 4217 log.cc:1196] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/4b91195197be4e97be4c56cfc07cf1a6
I20260807 08:29:32.634225 4217 ts_tablet_manager.cc:1939] T 4b91195197be4e97be4c56cfc07cf1a6 P 5f4e3679445345678f97ceae2fb22207: Deleting consensus metadata
I20260807 08:29:32.634747 4219 ts_tablet_manager.cc:1905] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:32.634842 3313 catalog_manager.cc:4841] TS 5f4e3679445345678f97ceae2fb22207 (127.1.157.67:41289): tablet 4b91195197be4e97be4c56cfc07cf1a6 (table test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]) successfully deleted
I20260807 08:29:32.635125 4210 ts_tablet_manager.cc:1918] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:32.635221 4210 log.cc:1196] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/4b91195197be4e97be4c56cfc07cf1a6
I20260807 08:29:32.635476 4210 ts_tablet_manager.cc:1939] T 4b91195197be4e97be4c56cfc07cf1a6 P ffcf5e2303944b54820aa14f647c9f72: Deleting consensus metadata
I20260807 08:29:32.636142 3312 catalog_manager.cc:4841] TS ffcf5e2303944b54820aa14f647c9f72 (127.1.157.66:36085): tablet 4b91195197be4e97be4c56cfc07cf1a6 (table test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]) successfully deleted
I20260807 08:29:32.638595 4219 ts_tablet_manager.cc:1918] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:32.638681 4219 log.cc:1196] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/4b91195197be4e97be4c56cfc07cf1a6
I20260807 08:29:32.639027 4219 ts_tablet_manager.cc:1939] T 4b91195197be4e97be4c56cfc07cf1a6 P 611ce7ba68c64984af55d6f064f343e0: Deleting consensus metadata
I20260807 08:29:32.639644 3314 catalog_manager.cc:4841] TS 611ce7ba68c64984af55d6f064f343e0 (127.1.157.65:41961): tablet 4b91195197be4e97be4c56cfc07cf1a6 (table test-table [id=e6d4f54362dc4ca484ac827e0dc97c13]) successfully deleted
Aug 07 08:29:32 dist-test-slave-6zbq krb5kdc[3267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091371, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:32.644498 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.631652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.1.157.65:41961)
0807 08:29:32.631766 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.631770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.635247 (+ 3477us) server_negotiation.cc:407] Connection header received
0807 08:29:32.635487 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.635491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.635539 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.635592 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:32.636606 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.637261 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.638044 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.638322 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.638442 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.640915 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:32.640925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:32.640939 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:32.640963 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:32.642767 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.643168 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.643181 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.643182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.643349 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.643613 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.643615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.643616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.643836 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:32.644022 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.644220 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.644327 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 08:29:32.644994 3655 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37886
I20260807 08:29:32.653443 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:32.646391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.1.157.126:34005)
0807 08:29:32.646500 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:32.646504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:32.646613 (+ 109us) server_negotiation.cc:407] Connection header received
0807 08:29:32.646726 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:32.646728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:32.646764 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:32.646824 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:32.647548 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.648041 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.649134 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:32.649340 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:32.649407 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:32.650821 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:32.650823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:32.650825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:32.650846 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:32.652057 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.652480 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.652482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.652483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.652547 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:32.652793 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:32.652795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:32.652796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:32.652943 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:32.653030 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:32.653245 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 08:29:32.653333 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 08:29:32.653745 3341 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36162
I20260807 08:29:32.654810 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3445
I20260807 08:29:32.670663 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3675
I20260807 08:29:32.683533 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3900
I20260807 08:29:32.691730 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3287
2026-08-07T08:29:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3154 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:32 dist-test-slave-6zbq krb5kdc[4231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:32 dist-test-slave-6zbq krb5kdc[4231](info): set up 2 sockets
Aug 07 08:29:32 dist-test-slave-6zbq krb5kdc[4231](info): commencing operation
krb5kdc: starting...
W20260807 08:29:33.813673 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:33 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:33.924580 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43405
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:34.029078 4250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:34.029314 4250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:34.029367 4250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:34.033177 4250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:34.033223 4250 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:29:34.033245 4250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:34.033278 4250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:34.033309 4250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:34.033337 4250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:34.038328 4250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43405
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:34.039220 4250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:34.040131 4250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:34.047776 4256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:34.047780 4255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.048000 4250 server_base.cc:1034] running on GCE node
W20260807 08:29:34.047816 4258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.048511 4250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:34.049448 4250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:34.050594 4250 hybrid_clock.cc:648] HybridClock initialized: now 1786091374050581 us; error 25 us; skew 500 ppm
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:34.053112 4250 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:34.053915 4250 webserver.cc:460] Webserver started at http://127.1.157.126:33395/ using document root <none> and password file <none>
I20260807 08:29:34.054432 4250 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:34.054497 4250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:34.054684 4250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:34.056547 4250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "f94596a30c134f83858295e96f79702a"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "2cee8ba430667409a7df8179c1dc1c8f"
server_key_iv: "b97396233cec68e97aee59a1a7d91faa"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.057061 4250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "f94596a30c134f83858295e96f79702a"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "2cee8ba430667409a7df8179c1dc1c8f"
server_key_iv: "b97396233cec68e97aee59a1a7d91faa"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.060683 4250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:29:34.063309 4268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.064442 4250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:34.064543 4250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "f94596a30c134f83858295e96f79702a"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "2cee8ba430667409a7df8179c1dc1c8f"
server_key_iv: "b97396233cec68e97aee59a1a7d91faa"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.064646 4250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:34.083527 4250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:34.086069 4250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:34.086306 4250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:34.093755 4250 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43405
I20260807 08:29:34.093762 4352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43405 every 8 connection(s)
I20260807 08:29:34.094623 4250 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:34.097261 4353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.099468 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 4250
I20260807 08:29:34.099596 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:34.099874 1653 external_mini_cluster.cc:1428] Setting key 06c4a18e1a4c5e238df5ab53ebf636a5
I20260807 08:29:34.103227 4353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Bootstrap starting.
I20260807 08:29:34.105623 4353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.106384 4353 log.cc:824] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.108508 4353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: No bootstrap required, opened a new log
I20260807 08:29:34.111865 4353 raft_consensus.cc:348] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } }
I20260807 08:29:34.112094 4353 raft_consensus.cc:374] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.112196 4353 raft_consensus.cc:729] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f94596a30c134f83858295e96f79702a, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.112668 4353 consensus_queue.cc:260] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } }
I20260807 08:29:34.112823 4353 raft_consensus.cc:388] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:34.112890 4353 raft_consensus.cc:482] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:34.112980 4353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:34.113513 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.101145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.1.157.126:43405)
0807 08:29:34.101524 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.101539 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.101572 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:29:34.102264 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.102280 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.102618 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.102889 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.103764 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.104710 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.105835 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.106283 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.106447 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.109001 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.109006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.109017 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.109042 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.110593 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.110993 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.110998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.111002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.111110 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.111386 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.111390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.111391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.111767 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.111945 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.112276 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.112565 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260807 08:29:34.113966 4353 raft_consensus.cc:504] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } }
I20260807 08:29:34.114285 4353 leader_election.cc:302] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f94596a30c134f83858295e96f79702a; no voters:
I20260807 08:29:34.114602 4353 leader_election.cc:288] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:34.114686 4361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:34.114853 4361 raft_consensus.cc:686] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [term 1 LEADER]: Becoming Leader. State: Replica: f94596a30c134f83858295e96f79702a, State: Running, Role: LEADER
I20260807 08:29:34.115128 4361 consensus_queue.cc:237] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } }
I20260807 08:29:34.115797 4353 sys_catalog.cc:564] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:34.116662 4362 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f94596a30c134f83858295e96f79702a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } } }
I20260807 08:29:34.116792 4362 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:34.117136 4374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:34.117095 4363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f94596a30c134f83858295e96f79702a. Latest consensus state: current_term: 1 leader_uuid: "f94596a30c134f83858295e96f79702a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f94596a30c134f83858295e96f79702a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43405 } } }
I20260807 08:29:34.117245 4363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:34.119879 4374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:34.124913 4374 catalog_manager.cc:1347] Generated new cluster ID: 45080914962c4b85bc60f6bcbd5ae6c3
I20260807 08:29:34.124974 4374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:34.134143 4374 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:34.134959 4374 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:34.152278 4374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Generated new TSK 0
I20260807 08:29:34.152865 4374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:34.179299 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43405
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:34.285312 4396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:34.285544 4396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:34.285606 4396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:34.289515 4396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:34.289565 4396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:34.289619 4396 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:29:34.289646 4396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:34.289693 4396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:34.294708 4396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:43405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:34.295680 4396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:34.296648 4396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:34.305560 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.305610 4396 server_base.cc:1034] running on GCE node
W20260807 08:29:34.305567 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:34.305588 4407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.306242 4396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:34.306861 4396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:34.308018 4396 hybrid_clock.cc:648] HybridClock initialized: now 1786091374307999 us; error 31 us; skew 500 ppm
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:34.310587 4396 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:34.311414 4396 webserver.cc:460] Webserver started at http://127.1.157.65:40673/ using document root <none> and password file <none>
I20260807 08:29:34.311919 4396 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:34.311978 4396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:34.312170 4396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:34.314100 4396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "72c65555454a4912862b23ad7895a118"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3ae032c303f51f71fd05383525e137d8"
server_key_iv: "9fc86473712612242806317899fe498c"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.314671 4396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "72c65555454a4912862b23ad7895a118"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3ae032c303f51f71fd05383525e137d8"
server_key_iv: "9fc86473712612242806317899fe498c"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.318287 4396 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:34.320472 4420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.321563 4396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:29:34.321656 4396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "72c65555454a4912862b23ad7895a118"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3ae032c303f51f71fd05383525e137d8"
server_key_iv: "9fc86473712612242806317899fe498c"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.321748 4396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:34.348769 4396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:34.351274 4396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:34.351434 4396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:34.352792 4396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:34.352844 4396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.352896 4396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:34.352921 4396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.363157 4396 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:33139
I20260807 08:29:34.363186 4592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:33139 every 8 connection(s)
I20260807 08:29:34.363960 4396 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:34.365034 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 4396
I20260807 08:29:34.365156 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:34.365463 1653 external_mini_cluster.cc:1428] Setting key 10ca18e929df355bd72f121f0fcb1df2
I20260807 08:29:34.365494 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.353603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:45915 (local address 127.1.157.126:43405)
0807 08:29:34.353747 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.353752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.354549 (+ 797us) server_negotiation.cc:407] Connection header received
0807 08:29:34.355486 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.355489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.355544 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.355600 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.356890 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.357706 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.358971 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.359272 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.359374 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.362082 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.362085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.362087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.362110 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.363539 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.363983 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.363987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.363989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.364073 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.364419 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.364422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.364424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.364637 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.364732 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.365275 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.365378 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:29:34.366276 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.353943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.65:45915)
0807 08:29:34.354398 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.354446 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.355265 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.355930 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.355935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.356288 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.356740 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.356759 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.357843 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.357847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.358856 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.358866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.359429 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.359433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.359597 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.360121 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.360142 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.361925 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.363640 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.363646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.363658 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.363891 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.364166 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.364169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.364178 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.364308 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.364743 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.364748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.365005 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.365222 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.365520 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
I20260807 08:29:34.367486 4593 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43405
I20260807 08:29:34.367620 4593 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:34.368052 4593 heartbeater.cc:511] Master 127.1.157.126:43405 requested a full tablet report, sending...
I20260807 08:29:34.370446 4290 ts_manager.cc:194] Registered new tserver with Master: 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139)
I20260807 08:29:34.371871 4290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:45915
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:34.378784 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.369739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:41639 (local address 127.1.157.126:43405)
0807 08:29:34.370450 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.370453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.370463 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:29:34.370489 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.370491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.370523 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.370589 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.371036 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.371681 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.374520 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.374855 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.374963 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.376037 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.376041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.376044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.376070 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.377433 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.377843 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.377845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.377847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.377917 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.378109 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.378111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.378112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.378272 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.378379 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.378590 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.378675 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":644}
I20260807 08:29:34.378790 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.369409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.65:41639)
0807 08:29:34.369681 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.369695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.369776 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.370700 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.370702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.370847 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.370935 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.370939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.371796 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.371800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.372432 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.372443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.375008 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.375012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.375145 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.375585 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.375595 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.375905 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.377562 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.377564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.377566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.377758 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.377984 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.377986 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.377987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.378050 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.378375 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.378377 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.378464 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.378545 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.378652 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:34.393674 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43405
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:34.498600 4606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:34.498809 4606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:34.498855 4606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:34.502717 4606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:34.502759 4606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:34.502795 4606 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:29:34.502823 4606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:34.502874 4606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:34.507719 4606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:43405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:34.508682 4606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:34.509575 4606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:34.518239 4614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:34.518258 4615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:34.518236 4617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.518505 4606 server_base.cc:1034] running on GCE node
I20260807 08:29:34.518937 4606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:34.519507 4606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:34.520665 4606 hybrid_clock.cc:648] HybridClock initialized: now 1786091374520651 us; error 26 us; skew 500 ppm
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:34.523139 4606 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:34.523876 4606 webserver.cc:460] Webserver started at http://127.1.157.66:39459/ using document root <none> and password file <none>
I20260807 08:29:34.524350 4606 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:34.524413 4606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:34.524581 4606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:34.526470 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "04df2566b588439eac7dba2f1847a495"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3a823b8a827ffc37474b5105e08a1537"
server_key_iv: "2b90bdd55f51b7d3bad5829890fed82b"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.526973 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "04df2566b588439eac7dba2f1847a495"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3a823b8a827ffc37474b5105e08a1537"
server_key_iv: "2b90bdd55f51b7d3bad5829890fed82b"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.530205 4606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:34.532312 4629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.533341 4606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:34.533442 4606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "04df2566b588439eac7dba2f1847a495"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "3a823b8a827ffc37474b5105e08a1537"
server_key_iv: "2b90bdd55f51b7d3bad5829890fed82b"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.533526 4606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:34.542989 4606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:34.545243 4606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:34.545400 4606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:34.546681 4606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:34.546730 4606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.546777 4606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:34.546806 4606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.556151 4606 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:33737
I20260807 08:29:34.556455 4796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:33737 every 8 connection(s)
I20260807 08:29:34.557039 4606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:34.558777 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 4606
I20260807 08:29:34.558884 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:34.559166 1653 external_mini_cluster.cc:1428] Setting key 10a811a0a855d61d6d617b2fcaa03f1d
I20260807 08:29:34.559193 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.547249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40665 (local address 127.1.157.126:43405)
0807 08:29:34.547393 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.547398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.548474 (+ 1076us) server_negotiation.cc:407] Connection header received
0807 08:29:34.549495 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.549498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.549557 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.549654 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.550836 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.551641 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.552930 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.553285 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.553386 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.555915 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.555919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.555922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.555950 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.557249 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.557801 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.557804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.557805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.557883 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.558148 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.558150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.558151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.558304 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.558429 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.558924 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.559045 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:29:34.559960 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.547928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.66:40665)
0807 08:29:34.548327 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.548373 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.549282 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.549822 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.549830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.550184 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.550696 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.550709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.551749 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.551753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.552815 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.552827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.553414 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.553418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.553600 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.554111 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.554125 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.555728 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.557357 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.557363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.557376 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.557697 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.557961 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.557964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.557965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.558090 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.558422 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.558427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.558668 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.558860 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.559152 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
I20260807 08:29:34.561002 4798 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43405
I20260807 08:29:34.561152 4798 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:34.561578 4798 heartbeater.cc:511] Master 127.1.157.126:43405 requested a full tablet report, sending...
I20260807 08:29:34.562552 4289 ts_manager.cc:194] Registered new tserver with Master: 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737)
I20260807 08:29:34.563253 4289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:40665
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:34.570147 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.563063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40819 (local address 127.1.157.126:43405)
0807 08:29:34.563160 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.563164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.563506 (+ 342us) server_negotiation.cc:407] Connection header received
0807 08:29:34.563610 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.563612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.563653 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.563731 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.564174 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.564888 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.565979 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.566180 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.566256 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.567282 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.567285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.567286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.567307 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.568697 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.569135 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.569137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.569138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.569201 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.569451 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.569453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.569454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.569629 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.569700 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.569963 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.570048 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:34.570210 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.563084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.66:40819)
0807 08:29:34.563437 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.563450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.563541 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.563749 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.563751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.563936 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.564056 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.564063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.565038 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.565042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.565869 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.565881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.566280 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.566284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.566393 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.566770 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.566784 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.567141 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.568814 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.568817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.568820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.569051 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.569281 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.569284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.569286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.569374 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.569712 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.569715 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.569810 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.569916 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.570036 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:34.585292 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43405
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:34.691622 4808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:34.691833 4808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:34.691875 4808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:34.695664 4808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:34.695712 4808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:34.695749 4808 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 08:29:34.695767 4808 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:29:34.695796 4808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:34.700579 4808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:43405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:34.701385 4808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:34.702200 4808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:34.710613 4816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.710690 4808 server_base.cc:1034] running on GCE node
W20260807 08:29:34.710642 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:34.710625 4814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:34.711143 4808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:34.711706 4808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:34.712859 4808 hybrid_clock.cc:648] HybridClock initialized: now 1786091374712848 us; error 37 us; skew 500 ppm
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:34.715556 4808 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:34.716348 4808 webserver.cc:460] Webserver started at http://127.1.157.67:35221/ using document root <none> and password file <none>
I20260807 08:29:34.716922 4808 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:34.716989 4808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:34.717180 4808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:34.719100 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "e4ca3f56ed23cae435a17f3ee902cabf"
server_key_iv: "1e9075be3ad96a56d35694e18a0e4272"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.719643 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "e4ca3f56ed23cae435a17f3ee902cabf"
server_key_iv: "1e9075be3ad96a56d35694e18a0e4272"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.723032 4808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:34.725158 4826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.726226 4808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:34.726377 4808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
format_stamp: "Formatted at 2026-08-07 08:29:34 on dist-test-slave-6zbq"
server_key: "e4ca3f56ed23cae435a17f3ee902cabf"
server_key_iv: "1e9075be3ad96a56d35694e18a0e4272"
server_key_version: "encryptionkey@0"
I20260807 08:29:34.726471 4808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:34.742818 4808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:34.745374 4808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:34.745532 4808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:34.746807 4808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:34.746855 4808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:34.746905 4808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:34.746934 4808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.756676 4808 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:36413
I20260807 08:29:34.756734 5003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:36413 every 8 connection(s)
I20260807 08:29:34.757450 4808 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:34.759831 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.747505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:54421 (local address 127.1.157.126:43405)
0807 08:29:34.747646 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.747651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.748408 (+ 757us) server_negotiation.cc:407] Connection header received
0807 08:29:34.749388 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.749391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.749444 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.749523 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.750951 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.751727 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.753309 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.753641 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.753786 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.756462 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.756466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.756469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.756499 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.757897 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.758440 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.758444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.758446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.758530 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.758843 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.758847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.758849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.759021 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.759105 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.759553 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.759673 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:29:34.760216 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 4808
I20260807 08:29:34.760344 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:34.760532 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.747837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.67:54421)
0807 08:29:34.748251 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.748297 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.749186 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.749798 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.749806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.750209 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.750814 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.750833 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.751848 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.751852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.753166 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.753178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.753801 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.753804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.753935 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.754454 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.754474 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.756278 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.758023 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.758029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.758041 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.758326 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.758616 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.758619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.758622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.758749 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.759115 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.759121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.759331 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.759473 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.759762 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260807 08:29:34.760591 1653 external_mini_cluster.cc:1428] Setting key cee0157cc709e0ce1f8b5514c328e095
I20260807 08:29:34.761785 5004 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43405
I20260807 08:29:34.761947 5004 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:34.762501 5004 heartbeater.cc:511] Master 127.1.157.126:43405 requested a full tablet report, sending...
I20260807 08:29:34.763446 4289 ts_manager.cc:194] Registered new tserver with Master: d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:34.764148 4289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:54421
I20260807 08:29:34.769968 5011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.763797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.1.157.67:43315)
0807 08:29:34.764138 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.764152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.764247 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.764511 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.764515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.764720 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.764844 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.764850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.765774 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.765776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.766305 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.766312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.766719 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.766721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.766782 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.767179 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.767192 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.767451 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.768900 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.768903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.768904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.769072 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.769290 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.769292 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.769294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.769363 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.769624 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.769626 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.769690 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.769759 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.769848 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":17,"thread_start_us":92,"threads_started":1}
I20260807 08:29:34.769989 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.763885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:43315 (local address 127.1.157.126:43405)
0807 08:29:34.763989 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.763992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.764221 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:29:34.764365 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.764368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.764414 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.764488 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.764949 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.765675 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.766431 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.766631 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.766702 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.767559 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.767562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.767565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.767587 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.768812 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.769146 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.769149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.769151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.769220 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.769418 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.769420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.769421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.769561 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.769629 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.769803 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.769884 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:34.774106 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.797504 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.790815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.1.157.126:43405)
0807 08:29:34.790915 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.790918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.790928 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:29:34.790955 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.790957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.790987 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.791045 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.791650 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.792107 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.792823 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.793011 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.793069 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.795034 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.795037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.795039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.795060 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.796209 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.796556 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.796559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.796560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.796631 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.796853 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.796856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.796858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.797021 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.797137 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.797273 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.797356 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:29:34.799861 4289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:34.802726 4289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:34.815577 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.810828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.1.157.66:33737)
0807 08:29:34.811234 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.811245 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.811265 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:29:34.811340 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.811345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.811516 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.811653 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:34.812146 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.813039 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.814081 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.814885 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.814969 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.815109 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.815355 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.815441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":95,"threads_started":1}
I20260807 08:29:34.815577 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.810272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.1.157.67:36413)
0807 08:29:34.810715 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.810726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.810748 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:29:34.810830 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.810835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.811028 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.811171 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:34.811454 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.812495 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.814042 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.814851 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.814959 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.815109 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.815355 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.815441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":151,"threads_started":1}
I20260807 08:29:34.815598 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.810135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36413 (local address 127.0.0.1:33930)
0807 08:29:34.810455 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.810496 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.810602 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.811158 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.811163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.811184 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:34.811322 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.811330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.812626 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.812629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.813929 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.813939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.814985 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.814989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.815156 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.815219 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.815325 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.815455 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":68,"thread_start_us":73,"threads_started":1}
I20260807 08:29:34.816147 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.810595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.0.0.1:37150)
0807 08:29:34.810918 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.810930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.811008 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.811914 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.811917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.811934 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:34.812030 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.812036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.813143 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.813147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.813977 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.813987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.815071 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.815075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.815160 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.815219 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.815330 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.815455 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":8,"thread_start_us":82,"threads_started":1}
I20260807 08:29:34.817078 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.810832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33139 (local address 127.0.0.1:45932)
0807 08:29:34.811704 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.811717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.811807 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.812354 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.812357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.812373 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:34.812469 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.812475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.813841 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.813845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.815488 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.815497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.816677 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.816681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.816776 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.816798 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.816855 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.816924 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":464,"thread_start_us":86,"threads_started":1}
I20260807 08:29:34.817189 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.811233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45932 (local address 127.1.157.65:33139)
0807 08:29:34.811929 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.811940 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.811961 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:29:34.812063 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.812068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.812241 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.812361 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:34.812566 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.813722 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.815613 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.816565 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.816665 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.816832 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.816892 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.817026 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":99,"threads_started":1}
I20260807 08:29:34.818602 4701 tablet_service.cc:1467] Processing CreateTablet for tablet e78e236be3e74e618adab23d94fd64d3 (DEFAULT_TABLE table=test-table [id=4521c213788a4426893cb2e3ebf2c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:34.818602 4907 tablet_service.cc:1467] Processing CreateTablet for tablet e78e236be3e74e618adab23d94fd64d3 (DEFAULT_TABLE table=test-table [id=4521c213788a4426893cb2e3ebf2c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:34.819664 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78e236be3e74e618adab23d94fd64d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.819669 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78e236be3e74e618adab23d94fd64d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.819860 4492 tablet_service.cc:1467] Processing CreateTablet for tablet e78e236be3e74e618adab23d94fd64d3 (DEFAULT_TABLE table=test-table [id=4521c213788a4426893cb2e3ebf2c3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:34.820670 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78e236be3e74e618adab23d94fd64d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.827579 5037 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Bootstrap starting.
I20260807 08:29:34.827697 5036 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Bootstrap starting.
I20260807 08:29:34.829715 5037 tablet_bootstrap.cc:654] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.829715 5036 tablet_bootstrap.cc:654] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.830127 5035 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Bootstrap starting.
I20260807 08:29:34.830581 5036 log.cc:824] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:34.830581 5037 log.cc:824] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:34.832216 5035 tablet_bootstrap.cc:654] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.832450 5036 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: No bootstrap required, opened a new log
I20260807 08:29:34.832450 5037 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: No bootstrap required, opened a new log
I20260807 08:29:34.832633 5037 ts_tablet_manager.cc:1397] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:29:34.832633 5036 ts_tablet_manager.cc:1397] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:29:34.832973 5035 log.cc:824] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:34.834561 5035 tablet_bootstrap.cc:492] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: No bootstrap required, opened a new log
I20260807 08:29:34.834733 5035 ts_tablet_manager.cc:1397] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:29:34.836408 5036 raft_consensus.cc:348] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.836386 5037 raft_consensus.cc:348] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.836642 5036 raft_consensus.cc:374] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.836643 5037 raft_consensus.cc:374] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.836699 5037 raft_consensus.cc:729] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.836699 5036 raft_consensus.cc:729] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.837142 5037 consensus_queue.cc:260] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.837142 5036 consensus_queue.cc:260] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.837873 5004 heartbeater.cc:503] Master 127.1.157.126:43405 was elected leader, sending a full tablet report...
I20260807 08:29:34.838232 5037 ts_tablet_manager.cc:1428] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:29:34.838253 5036 ts_tablet_manager.cc:1428] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:29:34.838485 4798 heartbeater.cc:503] Master 127.1.157.126:43405 was elected leader, sending a full tablet report...
I20260807 08:29:34.838600 5035 raft_consensus.cc:348] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.838855 5035 raft_consensus.cc:374] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.838912 5035 raft_consensus.cc:729] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c65555454a4912862b23ad7895a118, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.839295 5035 consensus_queue.cc:260] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.840046 4593 heartbeater.cc:503] Master 127.1.157.126:43405 was elected leader, sending a full tablet report...
I20260807 08:29:34.840247 5035 ts_tablet_manager.cc:1428] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260807 08:29:34.865000 4594 tablet.cc:2369] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:34.900490 5046 raft_consensus.cc:482] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:34.900660 5046 raft_consensus.cc:504] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.901719 5046 leader_election.cc:288] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413), 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737)
I20260807 08:29:34.905555 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.902035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:48019 (local address 127.1.157.67:36413)
0807 08:29:34.902156 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.902160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.902303 (+ 143us) server_negotiation.cc:407] Connection header received
0807 08:29:34.902466 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.902469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.902514 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.902578 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:34.902830 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.903334 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.904671 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.905050 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.905153 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.905214 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.905370 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.905434 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:29:34.905586 5047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.901913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36413 (local address 127.1.157.65:48019)
0807 08:29:34.902201 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.902215 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.902362 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.902596 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.902599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.902614 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:34.902733 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.902738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.903432 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.903435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.904556 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.904566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.905158 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.905161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.905256 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.905280 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.905346 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.905413 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":53,"thread_start_us":77,"threads_started":1}
I20260807 08:29:34.905596 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.902086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:48165 (local address 127.1.157.66:33737)
0807 08:29:34.902202 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.902206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.902363 (+ 157us) server_negotiation.cc:407] Connection header received
0807 08:29:34.902513 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.902516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.902564 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.902626 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:34.902916 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.903392 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.904674 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.905082 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.905187 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.905234 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.905411 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.905477 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:29:34.905615 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.901959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.1.157.65:48165)
0807 08:29:34.902257 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.902267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.902384 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.902689 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.902692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.902708 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:34.902807 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.902811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.903494 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.903497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.904561 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.904571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.905188 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.905191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.905297 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.905320 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.905384 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.905451 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":48,"thread_start_us":83,"threads_started":1}
I20260807 08:29:34.906112 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78e236be3e74e618adab23d94fd64d3" candidate_uuid: "72c65555454a4912862b23ad7895a118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" is_pre_election: true
I20260807 08:29:34.906112 4726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78e236be3e74e618adab23d94fd64d3" candidate_uuid: "72c65555454a4912862b23ad7895a118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df2566b588439eac7dba2f1847a495" is_pre_election: true
I20260807 08:29:34.906422 4935 raft_consensus.cc:2454] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72c65555454a4912862b23ad7895a118 in term 0.
I20260807 08:29:34.906422 4726 raft_consensus.cc:2454] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72c65555454a4912862b23ad7895a118 in term 0.
I20260807 08:29:34.906869 4424 leader_election.cc:302] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:34.907094 5046 raft_consensus.cc:2781] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:34.907171 5046 raft_consensus.cc:482] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:34.907217 5046 raft_consensus.cc:3037] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:34.908135 5046 raft_consensus.cc:504] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.908427 5046 leader_election.cc:288] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [CANDIDATE]: Term 1 election: Requested vote from peers d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413), 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737)
I20260807 08:29:34.908723 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78e236be3e74e618adab23d94fd64d3" candidate_uuid: "72c65555454a4912862b23ad7895a118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
I20260807 08:29:34.908757 4726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78e236be3e74e618adab23d94fd64d3" candidate_uuid: "72c65555454a4912862b23ad7895a118" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df2566b588439eac7dba2f1847a495"
I20260807 08:29:34.908835 4935 raft_consensus.cc:3037] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:34.908849 4726 raft_consensus.cc:3037] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:34.910136 4935 raft_consensus.cc:2454] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72c65555454a4912862b23ad7895a118 in term 1.
I20260807 08:29:34.910183 4726 raft_consensus.cc:2454] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72c65555454a4912862b23ad7895a118 in term 1.
I20260807 08:29:34.910439 4422 leader_election.cc:302] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72c65555454a4912862b23ad7895a118, d5f2869ee0ae4b7cbb81f92e757221d0; no voters:
I20260807 08:29:34.910594 5046 raft_consensus.cc:2781] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:34.910759 5046 raft_consensus.cc:686] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 1 LEADER]: Becoming Leader. State: Replica: 72c65555454a4912862b23ad7895a118, State: Running, Role: LEADER
I20260807 08:29:34.911052 5046 consensus_queue.cc:237] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } }
I20260807 08:29:34.913854 4288 catalog_manager.cc:5495] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72c65555454a4912862b23ad7895a118 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "72c65555454a4912862b23ad7895a118" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:29:34 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091374, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:34.955075 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.947668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43405 (local address 127.0.0.1:33696)
0807 08:29:34.948007 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:34.948018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:34.948117 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:34.948331 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:34.948334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:34.948719 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:34.948810 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.948817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.949524 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.949527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.950072 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:34.950081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.950525 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.950528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:34.950617 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.951125 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:34.951136 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:34.952476 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:34.953790 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.953799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.953803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.954017 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.954214 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:34.954216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:34.954217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:34.954287 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:34.954610 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:34.954615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:34.954712 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 08:29:34.954798 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:29:34.954908 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":158,"threads_started":1}
I20260807 08:29:34.955082 4358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:34.947732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.1.157.126:43405)
0807 08:29:34.947838 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:34.947842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:34.948110 (+ 268us) server_negotiation.cc:407] Connection header received
0807 08:29:34.948200 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:34.948203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:34.948239 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:34.948302 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:34.948930 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.949427 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.950183 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:34.950420 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:34.950504 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:34.952593 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:34.952596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:34.952598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:34.952619 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:34.953693 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.954086 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.954089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.954090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.954154 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:34.954373 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:34.954375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:34.954377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:34.954533 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:34.954606 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:34.954838 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 08:29:34.954960 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":39}
I20260807 08:29:34.958366 4288 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33696:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:29:34.959017 4288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:34.963555 4492 tablet_service.cc:1467] Processing CreateTablet for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=433f3fcb443c419b993c693b4bd459c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:34.963706 4701 tablet_service.cc:1467] Processing CreateTablet for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=433f3fcb443c419b993c693b4bd459c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:34.963724 4907 tablet_service.cc:1467] Processing CreateTablet for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=433f3fcb443c419b993c693b4bd459c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:29:34.964255 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.964258 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.964257 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5c0c4c20604dd38a8c24e73f3bb9b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:34.966629 5037 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: Bootstrap starting.
I20260807 08:29:34.966980 5036 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: Bootstrap starting.
I20260807 08:29:34.966980 5035 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: Bootstrap starting.
I20260807 08:29:34.967588 5037 tablet_bootstrap.cc:654] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.967922 5036 tablet_bootstrap.cc:654] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.967924 5035 tablet_bootstrap.cc:654] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:34.969204 5036 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: No bootstrap required, opened a new log
I20260807 08:29:34.969296 5036 ts_tablet_manager.cc:1397] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:34.969444 5035 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: No bootstrap required, opened a new log
I20260807 08:29:34.969519 5035 ts_tablet_manager.cc:1397] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:34.969612 5037 tablet_bootstrap.cc:492] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: No bootstrap required, opened a new log
I20260807 08:29:34.969699 5037 ts_tablet_manager.cc:1397] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:34.969743 5036 raft_consensus.cc:348] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.969866 5036 raft_consensus.cc:374] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.969901 5036 raft_consensus.cc:729] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.969933 5035 raft_consensus.cc:348] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.970041 5035 raft_consensus.cc:374] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.970021 5036 consensus_queue.cc:260] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.970081 5035 raft_consensus.cc:729] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c65555454a4912862b23ad7895a118, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.970126 5037 raft_consensus.cc:348] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.970225 5037 raft_consensus.cc:374] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:34.970201 5035 consensus_queue.cc:260] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.970273 5037 raft_consensus.cc:729] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Initialized, Role: FOLLOWER
I20260807 08:29:34.970279 5036 ts_tablet_manager.cc:1428] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:29:34.970470 5035 ts_tablet_manager.cc:1428] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:34.970438 5037 consensus_queue.cc:260] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:34.970686 5037 ts_tablet_manager.cc:1428] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:34.970917 5045 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:34.971058 5045 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:34.971242 5046 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:34.971390 5046 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:34.971589 5043 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:34.971719 5043 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 08:29:35.008381 5005 tablet.cc:2369] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:35.058037 4799 tablet.cc:2369] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:35.058525 5045 raft_consensus.cc:482] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:35.058619 5045 raft_consensus.cc:504] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.059597 5045 leader_election.cc:288] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:35.064282 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.059777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33139 (local address 127.1.157.66:58471)
0807 08:29:35.059887 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.059901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.059985 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.060253 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.060256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.060276 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:35.060397 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.060403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.061449 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.061452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.062793 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.062805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.063575 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.063579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.063964 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.064016 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.064078 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.064154 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":61}
I20260807 08:29:35.064365 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.060106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:48971 (local address 127.1.157.67:36413)
0807 08:29:35.060217 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.060221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.060337 (+ 116us) server_negotiation.cc:407] Connection header received
0807 08:29:35.060425 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.060427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.060466 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.060541 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:35.060808 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.061530 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.062950 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.063506 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.063623 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.063685 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.064129 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.064234 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:29:35.064738 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.059863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58471 (local address 127.1.157.65:33139)
0807 08:29:35.059972 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.059977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.059992 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:29:35.060073 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.060076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.060123 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.060197 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:35.060493 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.061350 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.062956 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.063460 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.063575 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.063630 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.064532 (+ 902us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.064613 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:29:35.064766 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.060021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36413 (local address 127.1.157.66:48971)
0807 08:29:35.060253 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.060265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.060354 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.060551 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.060553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.060568 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:35.060677 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.060682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.061615 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.061618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.062824 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.062832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.063729 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.063735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.063969 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.064005 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.064532 (+ 527us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.064643 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":77,"thread_start_us":72,"threads_started":1}
I20260807 08:29:35.065516 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5c0c4c20604dd38a8c24e73f3bb9b2" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118" is_pre_election: true
I20260807 08:29:35.065693 4519 raft_consensus.cc:2454] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:35.066093 4631 leader_election.cc:302] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:35.066303 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5c0c4c20604dd38a8c24e73f3bb9b2" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" is_pre_election: true
I20260807 08:29:35.066380 5045 raft_consensus.cc:2781] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:35.066432 5045 raft_consensus.cc:482] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:35.066476 5045 raft_consensus.cc:3037] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.066460 4935 raft_consensus.cc:2454] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:35.067528 5045 raft_consensus.cc:504] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.067838 5045 leader_election.cc:288] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Requested vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:35.068197 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5c0c4c20604dd38a8c24e73f3bb9b2" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
I20260807 08:29:35.068222 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5c0c4c20604dd38a8c24e73f3bb9b2" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118"
I20260807 08:29:35.068306 4935 raft_consensus.cc:3037] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.068343 4519 raft_consensus.cc:3037] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.069190 4935 raft_consensus.cc:2454] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:35.069312 4519 raft_consensus.cc:2454] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:35.069437 4631 leader_election.cc:302] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, d5f2869ee0ae4b7cbb81f92e757221d0; no voters:
I20260807 08:29:35.069595 5045 raft_consensus.cc:2781] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:35.069811 5045 raft_consensus.cc:686] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [term 1 LEADER]: Becoming Leader. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Running, Role: LEADER
I20260807 08:29:35.070120 5045 consensus_queue.cc:237] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.072317 5067 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: TxnStatusTablet state changed. Reason: New leader 04df2566b588439eac7dba2f1847a495. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.072427 5067 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:35.072916 5070 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:29:35.073140 5070 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:29:35.073061 4288 catalog_manager.cc:5495] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04df2566b588439eac7dba2f1847a495 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:35.099680 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.095536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.1.157.66:33737)
0807 08:29:35.095647 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.095651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.095664 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:29:35.095812 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.095815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.095869 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.095959 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.096187 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.096937 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.098121 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.098477 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.098569 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.098876 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.099315 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.099406 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.099521 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.099568 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:29:35.099711 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.095442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.0.0.1:37152)
0807 08:29:35.095572 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.095586 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.095670 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.095982 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.095985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.095993 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:35.096093 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.096099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.097043 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.097046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.098000 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.098010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.098582 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.098586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.098694 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.098745 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.099414 (+ 669us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.099426 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.099490 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.099571 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:29:35.105415 4935 raft_consensus.cc:1261] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.105525 4519 raft_consensus.cc:1261] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.106033 5067 consensus_queue.cc:1035] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:35.106271 5045 consensus_queue.cc:1035] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:35.109402 5046 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: TxnStatusTablet state changed. Reason: New leader 04df2566b588439eac7dba2f1847a495. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.109505 5046 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:35.109649 5043 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: TxnStatusTablet state changed. Reason: New leader 04df2566b588439eac7dba2f1847a495. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.109735 5043 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:35.112970 5045 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.113077 5045 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:35.113021 5067 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.113131 5067 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 04df2566b588439eac7dba2f1847a495: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:29:35.114872 5043 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.114989 5043 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P d5f2869ee0ae4b7cbb81f92e757221d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:35.115701 5079 mvcc.cc:205] Tried to move back new op lower bound from 7315830272426020864 to 7315830272290197504. Current Snapshot: MvccSnapshot[applied={T|T < 7315830272426020864}]
I20260807 08:29:35.115707 5046 tablet_replica.cc:439] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } } }
I20260807 08:29:35.115794 5046 tablet_replica.cc:442] T ad5c0c4c20604dd38a8c24e73f3bb9b2 P 72c65555454a4912862b23ad7895a118: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:29:35.116146 5080 mvcc.cc:205] Tried to move back new op lower bound from 7315830272426020864 to 7315830272290197504. Current Snapshot: MvccSnapshot[applied={T|T < 7315830272426020864}]
I20260807 08:29:35.124089 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.120553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.1.157.65:33139)
0807 08:29:35.120661 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.120665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.120678 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:29:35.120794 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.120797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.120850 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.120952 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.121139 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.121885 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.122738 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.123075 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.123179 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.123368 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.123788 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.123894 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.123925 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.123975 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:29:35.133831 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.130614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:56667 (local address 127.1.157.66:33737)
0807 08:29:35.130729 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.130733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.130897 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:29:35.131003 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.131006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.131054 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.131128 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.131344 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.131813 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.132977 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.133173 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.133233 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.133523 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.133577 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.133621 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.133716 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.133749 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:29:35.133931 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.130552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.1.157.65:56667)
0807 08:29:35.130830 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.130843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.130936 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.131147 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.131149 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.131156 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:35.131271 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.131277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.131904 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.131907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.132877 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.132887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.133258 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.133262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.133354 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.133401 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.133644 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.133650 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.133711 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.133782 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
I20260807 08:29:35.139739 4935 raft_consensus.cc:1261] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Refusing update from remote peer 72c65555454a4912862b23ad7895a118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.139874 4726 raft_consensus.cc:1261] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Refusing update from remote peer 72c65555454a4912862b23ad7895a118: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.140197 5046 consensus_queue.cc:1035] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:35.140349 5049 consensus_queue.cc:1035] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:35.161675 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.158134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:37211 (local address 127.1.157.65:33139)
0807 08:29:35.158230 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.158234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.158464 (+ 230us) server_negotiation.cc:407] Connection header received
0807 08:29:35.158570 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.158573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.158614 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.158673 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.158883 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.159505 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.160718 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.160929 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.161037 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.161312 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.161371 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.161422 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.161517 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.161565 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:29:35.161708 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.158078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33139 (local address 127.1.157.66:37211)
0807 08:29:35.158387 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.158401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.158498 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.158688 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.158691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.158698 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:35.158792 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.158798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.159627 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.159631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.160610 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.160622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.161030 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.161033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.161151 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.161199 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.161441 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.161446 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.161504 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.161576 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260807 08:29:35.183930 4288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:35.184083 4288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:35.186506 4288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Sending DeleteTablet for 3 replicas of tablet e78e236be3e74e618adab23d94fd64d3
I20260807 08:29:35.187244 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet e78e236be3e74e618adab23d94fd64d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:35 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:45932
I20260807 08:29:35.187337 4701 tablet_service.cc:1514] Processing DeleteTablet for tablet e78e236be3e74e618adab23d94fd64d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:35 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37150
I20260807 08:29:35.187650 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet e78e236be3e74e618adab23d94fd64d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:35 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33930
I20260807 08:29:35.187737 5109 tablet_replica.cc:330] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: stopping tablet replica
I20260807 08:29:35.187992 5109 raft_consensus.cc:2229] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:35.188136 5109 raft_consensus.cc:2258] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:35.188602 5108 tablet_replica.cc:330] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: stopping tablet replica
I20260807 08:29:35.188855 5108 raft_consensus.cc:2229] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:35.188871 4288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:29:35.189157 5108 raft_consensus.cc:2258] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 08:29:35.189230 4288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:35.188427 5110 tablet_replica.cc:330] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: stopping tablet replica
I20260807 08:29:35.189608 5110 raft_consensus.cc:2229] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:35.189785 5110 raft_consensus.cc:2258] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:35.191156 5109 ts_tablet_manager.cc:1905] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:35.194197 5109 ts_tablet_manager.cc:1918] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:35.194294 5109 log.cc:1196] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/e78e236be3e74e618adab23d94fd64d3
I20260807 08:29:35.194428 4701 tablet_service.cc:1467] Processing CreateTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff (DEFAULT_TABLE table=test-table [id=fa7603cca009488ea8f21418544df5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:35.194615 5109 ts_tablet_manager.cc:1939] T e78e236be3e74e618adab23d94fd64d3 P 04df2566b588439eac7dba2f1847a495: Deleting consensus metadata
I20260807 08:29:35.194665 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc097a22ee0a4cba94d90c4066ad3dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:35.195700 4275 catalog_manager.cc:4841] TS 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737): tablet e78e236be3e74e618adab23d94fd64d3 (table test-table [id=4521c213788a4426893cb2e3ebf2c3ef]) successfully deleted
I20260807 08:29:35.196686 5036 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Bootstrap starting.
I20260807 08:29:35.196703 5110 ts_tablet_manager.cc:1905] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:35.197562 5036 tablet_bootstrap.cc:654] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:35.198968 4492 tablet_service.cc:1467] Processing CreateTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff (DEFAULT_TABLE table=test-table [id=fa7603cca009488ea8f21418544df5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:35.199126 4907 tablet_service.cc:1467] Processing CreateTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff (DEFAULT_TABLE table=test-table [id=fa7603cca009488ea8f21418544df5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:35.199206 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc097a22ee0a4cba94d90c4066ad3dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:35.199327 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc097a22ee0a4cba94d90c4066ad3dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:35.199561 5110 ts_tablet_manager.cc:1918] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:35.199644 5110 log.cc:1196] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/e78e236be3e74e618adab23d94fd64d3
I20260807 08:29:35.199932 5110 ts_tablet_manager.cc:1939] T e78e236be3e74e618adab23d94fd64d3 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting consensus metadata
I20260807 08:29:35.201006 4271 catalog_manager.cc:4841] TS d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413): tablet e78e236be3e74e618adab23d94fd64d3 (table test-table [id=4521c213788a4426893cb2e3ebf2c3ef]) successfully deleted
I20260807 08:29:35.201545 5037 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Bootstrap starting.
I20260807 08:29:35.202406 5037 tablet_bootstrap.cc:654] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:35.202497 5035 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Bootstrap starting.
I20260807 08:29:35.203379 5035 tablet_bootstrap.cc:654] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:35.202474 5108 ts_tablet_manager.cc:1905] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:35.203482 5037 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: No bootstrap required, opened a new log
I20260807 08:29:35.203558 5037 ts_tablet_manager.cc:1397] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:35.204231 5037 raft_consensus.cc:348] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.204360 5037 raft_consensus.cc:374] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:35.204401 5037 raft_consensus.cc:729] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Initialized, Role: FOLLOWER
I20260807 08:29:35.204514 5037 consensus_queue.cc:260] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.204771 5037 ts_tablet_manager.cc:1428] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 08:29:35.205834 5035 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: No bootstrap required, opened a new log
I20260807 08:29:35.205917 5035 ts_tablet_manager.cc:1397] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:35.206370 5035 raft_consensus.cc:348] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.206475 5035 raft_consensus.cc:374] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:35.206507 5035 raft_consensus.cc:729] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c65555454a4912862b23ad7895a118, State: Initialized, Role: FOLLOWER
I20260807 08:29:35.206610 5035 consensus_queue.cc:260] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.206854 5035 ts_tablet_manager.cc:1428] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:35.209131 5108 ts_tablet_manager.cc:1918] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:35.209215 5108 log.cc:1196] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/e78e236be3e74e618adab23d94fd64d3
I20260807 08:29:35.209537 5108 ts_tablet_manager.cc:1939] T e78e236be3e74e618adab23d94fd64d3 P 72c65555454a4912862b23ad7895a118: Deleting consensus metadata
I20260807 08:29:35.210517 4271 catalog_manager.cc:4841] TS 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139): tablet e78e236be3e74e618adab23d94fd64d3 (table test-table [id=4521c213788a4426893cb2e3ebf2c3ef]) successfully deleted
I20260807 08:29:35.210985 5036 tablet_bootstrap.cc:492] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: No bootstrap required, opened a new log
I20260807 08:29:35.211062 5036 ts_tablet_manager.cc:1397] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20260807 08:29:35.211403 5036 raft_consensus.cc:348] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.211500 5036 raft_consensus.cc:374] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:35.211529 5036 raft_consensus.cc:729] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Initialized, Role: FOLLOWER
I20260807 08:29:35.211640 5036 consensus_queue.cc:260] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.211885 5036 ts_tablet_manager.cc:1428] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:35.383230 5085 raft_consensus.cc:482] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:35.383370 5085 raft_consensus.cc:504] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.383788 5085 leader_election.cc:288] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:35.384227 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc097a22ee0a4cba94d90c4066ad3dff" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118" is_pre_election: true
I20260807 08:29:35.384300 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc097a22ee0a4cba94d90c4066ad3dff" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" is_pre_election: true
I20260807 08:29:35.384377 4519 raft_consensus.cc:2454] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:35.384450 4935 raft_consensus.cc:2454] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:35.384670 4631 leader_election.cc:302] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:35.384842 5085 raft_consensus.cc:2781] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:35.384904 5085 raft_consensus.cc:482] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:35.384943 5085 raft_consensus.cc:3037] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.385897 5085 raft_consensus.cc:504] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.386181 5085 leader_election.cc:288] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Requested vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:35.386559 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc097a22ee0a4cba94d90c4066ad3dff" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118"
I20260807 08:29:35.386602 4935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc097a22ee0a4cba94d90c4066ad3dff" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
I20260807 08:29:35.386664 4519 raft_consensus.cc:3037] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.386718 4935 raft_consensus.cc:3037] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:35.387512 4519 raft_consensus.cc:2454] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:35.387779 4631 leader_election.cc:302] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:35.387833 4935 raft_consensus.cc:2454] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:35.387964 5085 raft_consensus.cc:2781] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:35.388051 5085 raft_consensus.cc:686] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 1 LEADER]: Becoming Leader. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Running, Role: LEADER
I20260807 08:29:35.388197 5085 consensus_queue.cc:237] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:35.389894 4289 catalog_manager.cc:5495] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04df2566b588439eac7dba2f1847a495 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:35.445602 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.442978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.1.157.66:33737)
0807 08:29:35.443085 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.443089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.443151 (+ 62us) server_negotiation.cc:407] Connection header received
0807 08:29:35.443258 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.443260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.443302 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.443355 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.443529 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.444008 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.444718 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.444912 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.444980 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.445160 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.445224 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.445298 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.445430 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.445477 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:29:35.453097 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47117 (local address 127.1.157.66:33737)
0807 08:29:35.450646 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:35.450650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:35.450662 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:35.450771 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:35.450774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:35.450813 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:35.450869 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:35.451043 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.451493 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.452227 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.452429 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.452508 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.452703 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.452768 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.452825 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:35.452915 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 08:29:35.452960 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:29:35.453097 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:35.450489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.1.157.66:47117)
0807 08:29:35.450597 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:35.450610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:35.450684 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:35.450884 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:35.450886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:35.450892 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:35.450963 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.450967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.451589 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.451592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.452130 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:35.452137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:35.452520 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:35.452523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:35.452592 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:35.452612 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:35.452840 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:35.452846 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:29:35.452901 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 08:29:35.452987 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260807 08:29:35.457690 4935 raft_consensus.cc:1261] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.457715 4519 raft_consensus.cc:1261] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:35.458103 5067 consensus_queue.cc:1035] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:35.458257 5045 consensus_queue.cc:1035] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:29:35.471874 5077 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:29:37.031929 4288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:37.032073 4288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:37.034299 4288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Sending DeleteTablet for 3 replicas of tablet bc097a22ee0a4cba94d90c4066ad3dff
I20260807 08:29:37.034793 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:45932
I20260807 08:29:37.034849 4701 tablet_service.cc:1514] Processing DeleteTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37150
I20260807 08:29:37.034891 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet bc097a22ee0a4cba94d90c4066ad3dff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33930
I20260807 08:29:37.035171 5151 tablet_replica.cc:330] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: stopping tablet replica
I20260807 08:29:37.035171 5153 tablet_replica.cc:330] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: stopping tablet replica
I20260807 08:29:37.035295 5153 raft_consensus.cc:2229] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:37.035354 5151 raft_consensus.cc:2229] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:37.035440 5153 raft_consensus.cc:2258] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.035697 5151 raft_consensus.cc:2258] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.036159 5153 ts_tablet_manager.cc:1905] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:37.037941 5151 ts_tablet_manager.cc:1905] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:37.039074 5153 ts_tablet_manager.cc:1918] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:29:37.039054 5150 tablet_replica.cc:330] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: stopping tablet replica
I20260807 08:29:37.039211 5150 raft_consensus.cc:2229] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:37.039222 5153 log.cc:1196] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/bc097a22ee0a4cba94d90c4066ad3dff
I20260807 08:29:37.039402 5150 raft_consensus.cc:2258] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.039472 5153 ts_tablet_manager.cc:1939] T bc097a22ee0a4cba94d90c4066ad3dff P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting consensus metadata
I20260807 08:29:37.040055 5150 ts_tablet_manager.cc:1905] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:37.040130 4271 catalog_manager.cc:4841] TS d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413): tablet bc097a22ee0a4cba94d90c4066ad3dff (table test-table [id=fa7603cca009488ea8f21418544df5e2]) successfully deleted
I20260807 08:29:37.040452 5151 ts_tablet_manager.cc:1918] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:29:37.040518 5151 log.cc:1196] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/bc097a22ee0a4cba94d90c4066ad3dff
I20260807 08:29:37.040781 5151 ts_tablet_manager.cc:1939] T bc097a22ee0a4cba94d90c4066ad3dff P 04df2566b588439eac7dba2f1847a495: Deleting consensus metadata
I20260807 08:29:37.041359 4275 catalog_manager.cc:4841] TS 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737): tablet bc097a22ee0a4cba94d90c4066ad3dff (table test-table [id=fa7603cca009488ea8f21418544df5e2]) successfully deleted
I20260807 08:29:37.042421 5150 ts_tablet_manager.cc:1918] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:29:37.042492 5150 log.cc:1196] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/bc097a22ee0a4cba94d90c4066ad3dff
I20260807 08:29:37.042742 5150 ts_tablet_manager.cc:1939] T bc097a22ee0a4cba94d90c4066ad3dff P 72c65555454a4912862b23ad7895a118: Deleting consensus metadata
I20260807 08:29:37.043427 4271 catalog_manager.cc:4841] TS 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139): tablet bc097a22ee0a4cba94d90c4066ad3dff (table test-table [id=fa7603cca009488ea8f21418544df5e2]) successfully deleted
Aug 07 08:29:37 dist-test-slave-6zbq krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:29:37 dist-test-slave-6zbq krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091377, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:37.067080 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.055596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.1.157.126:43405)
0807 08:29:37.055877 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.055884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.055898 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:29:37.055995 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.055997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.056038 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.056111 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:37.056707 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.057280 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.058067 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.058285 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.058413 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.064401 (+ 5988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:37.064404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:37.064406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:37.064429 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:37.065772 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:37.066075 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:37.066077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:37.066078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:37.066146 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:37.066369 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:37.066372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:37.066373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:37.066540 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:37.066640 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.066814 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.066908 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260807 08:29:37.069747 4288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:37.070204 4288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:37.074741 4492 tablet_service.cc:1467] Processing CreateTablet for tablet 404585f38e9340029df8c63a7a982a4d (DEFAULT_TABLE table=test-table [id=e7402a0a67c1414a9630235cbe613845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.074841 4701 tablet_service.cc:1467] Processing CreateTablet for tablet 404585f38e9340029df8c63a7a982a4d (DEFAULT_TABLE table=test-table [id=e7402a0a67c1414a9630235cbe613845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.075035 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404585f38e9340029df8c63a7a982a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.075120 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404585f38e9340029df8c63a7a982a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.075119 4907 tablet_service.cc:1467] Processing CreateTablet for tablet 404585f38e9340029df8c63a7a982a4d (DEFAULT_TABLE table=test-table [id=e7402a0a67c1414a9630235cbe613845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.075378 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404585f38e9340029df8c63a7a982a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.077044 5168 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Bootstrap starting.
I20260807 08:29:37.077719 5171 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Bootstrap starting.
I20260807 08:29:37.077940 5168 tablet_bootstrap.cc:654] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.078541 5173 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Bootstrap starting.
I20260807 08:29:37.078706 5171 tablet_bootstrap.cc:654] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.079653 5173 tablet_bootstrap.cc:654] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.079829 5171 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: No bootstrap required, opened a new log
I20260807 08:29:37.079911 5171 ts_tablet_manager.cc:1397] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:37.080377 5171 raft_consensus.cc:348] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.080477 5171 raft_consensus.cc:374] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.080507 5171 raft_consensus.cc:729] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c65555454a4912862b23ad7895a118, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.080657 5171 consensus_queue.cc:260] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.081084 5171 ts_tablet_manager.cc:1428] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:37.081655 5173 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: No bootstrap required, opened a new log
I20260807 08:29:37.081741 5173 ts_tablet_manager.cc:1397] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:37.082571 5173 raft_consensus.cc:348] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.082664 5173 raft_consensus.cc:374] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.082692 5173 raft_consensus.cc:729] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.082821 5173 consensus_queue.cc:260] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.083391 5173 ts_tablet_manager.cc:1428] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:37.083886 5168 tablet_bootstrap.cc:492] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: No bootstrap required, opened a new log
I20260807 08:29:37.083979 5168 ts_tablet_manager.cc:1397] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 08:29:37.084323 5168 raft_consensus.cc:348] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.084411 5168 raft_consensus.cc:374] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.084439 5168 raft_consensus.cc:729] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.084558 5168 consensus_queue.cc:260] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.084777 5168 ts_tablet_manager.cc:1428] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:29:37.212914 5123 raft_consensus.cc:482] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:37.213053 5123 raft_consensus.cc:504] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.213426 5123 leader_election.cc:288] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:37.213873 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404585f38e9340029df8c63a7a982a4d" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118" is_pre_election: true
I20260807 08:29:37.213999 4933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404585f38e9340029df8c63a7a982a4d" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" is_pre_election: true
I20260807 08:29:37.214049 4519 raft_consensus.cc:2454] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:37.214148 4933 raft_consensus.cc:2454] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 0.
I20260807 08:29:37.214422 4631 leader_election.cc:302] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:37.214607 5123 raft_consensus.cc:2781] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:37.214668 5123 raft_consensus.cc:482] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:37.214713 5123 raft_consensus.cc:3037] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.215854 5123 raft_consensus.cc:504] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.216188 5123 leader_election.cc:288] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Requested vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413)
I20260807 08:29:37.216483 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404585f38e9340029df8c63a7a982a4d" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118"
I20260807 08:29:37.216598 4519 raft_consensus.cc:3037] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.216737 4933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404585f38e9340029df8c63a7a982a4d" candidate_uuid: "04df2566b588439eac7dba2f1847a495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0"
I20260807 08:29:37.216878 4933 raft_consensus.cc:3037] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.217576 4519 raft_consensus.cc:2454] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:37.217840 4631 leader_election.cc:302] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, 72c65555454a4912862b23ad7895a118; no voters:
I20260807 08:29:37.218019 5123 raft_consensus.cc:2781] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:37.218111 5123 raft_consensus.cc:686] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 1 LEADER]: Becoming Leader. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Running, Role: LEADER
I20260807 08:29:37.218240 5123 consensus_queue.cc:237] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.218310 4933 raft_consensus.cc:2454] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df2566b588439eac7dba2f1847a495 in term 1.
I20260807 08:29:37.219911 4288 catalog_manager.cc:5495] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04df2566b588439eac7dba2f1847a495 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "04df2566b588439eac7dba2f1847a495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:37.236703 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.233363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.1.157.66:33737)
0807 08:29:37.233655 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.233692 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.233704 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:37.233738 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.233741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.233786 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.233857 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:37.234185 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.234788 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.235618 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.235845 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.235929 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.236260 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.236331 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.236390 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.236495 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.236549 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260807 08:29:37.242177 4518 raft_consensus.cc:1261] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:37.242481 4933 raft_consensus.cc:1261] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Refusing update from remote peer 04df2566b588439eac7dba2f1847a495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:37.242517 5123 consensus_queue.cc:1035] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:37.242830 5123 consensus_queue.cc:1035] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:37.270315 4289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:37.270478 4289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:37.272204 4289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Sending DeleteTablet for 3 replicas of tablet 404585f38e9340029df8c63a7a982a4d
I20260807 08:29:37.272706 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet 404585f38e9340029df8c63a7a982a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:45932
I20260807 08:29:37.272722 4701 tablet_service.cc:1514] Processing DeleteTablet for tablet 404585f38e9340029df8c63a7a982a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37150
I20260807 08:29:37.272728 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet 404585f38e9340029df8c63a7a982a4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:37 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33930
I20260807 08:29:37.272882 5151 tablet_replica.cc:330] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: stopping tablet replica
I20260807 08:29:37.272909 5150 tablet_replica.cc:330] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: stopping tablet replica
I20260807 08:29:37.272992 5151 raft_consensus.cc:2229] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:37.273031 5150 raft_consensus.cc:2229] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:37.273152 5150 raft_consensus.cc:2258] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.273169 5151 raft_consensus.cc:2258] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.273329 5153 tablet_replica.cc:330] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: stopping tablet replica
I20260807 08:29:37.273444 5153 raft_consensus.cc:2229] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:37.273551 5153 raft_consensus.cc:2258] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:37.273532 4289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:29:37.273864 5151 ts_tablet_manager.cc:1905] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:29:37.273904 4289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:37.274135 5150 ts_tablet_manager.cc:1905] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:37.274389 5153 ts_tablet_manager.cc:1905] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:37.276619 5153 ts_tablet_manager.cc:1918] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:37.276620 5151 ts_tablet_manager.cc:1918] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:37.276690 5151 log.cc:1196] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/404585f38e9340029df8c63a7a982a4d
I20260807 08:29:37.276688 5153 log.cc:1196] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/404585f38e9340029df8c63a7a982a4d
I20260807 08:29:37.276913 5150 ts_tablet_manager.cc:1918] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:29:37.276934 5153 ts_tablet_manager.cc:1939] T 404585f38e9340029df8c63a7a982a4d P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting consensus metadata
I20260807 08:29:37.276934 5151 ts_tablet_manager.cc:1939] T 404585f38e9340029df8c63a7a982a4d P 04df2566b588439eac7dba2f1847a495: Deleting consensus metadata
I20260807 08:29:37.276985 5150 log.cc:1196] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/404585f38e9340029df8c63a7a982a4d
I20260807 08:29:37.277225 5150 ts_tablet_manager.cc:1939] T 404585f38e9340029df8c63a7a982a4d P 72c65555454a4912862b23ad7895a118: Deleting consensus metadata
I20260807 08:29:37.277485 4275 catalog_manager.cc:4841] TS 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737): tablet 404585f38e9340029df8c63a7a982a4d (table test-table [id=e7402a0a67c1414a9630235cbe613845]) successfully deleted
I20260807 08:29:37.277485 4271 catalog_manager.cc:4841] TS d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413): tablet 404585f38e9340029df8c63a7a982a4d (table test-table [id=e7402a0a67c1414a9630235cbe613845]) successfully deleted
I20260807 08:29:37.277865 4271 catalog_manager.cc:4841] TS 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139): tablet 404585f38e9340029df8c63a7a982a4d (table test-table [id=e7402a0a67c1414a9630235cbe613845]) successfully deleted
I20260807 08:29:37.278435 4701 tablet_service.cc:1467] Processing CreateTablet for tablet 228df9ae0d2c4354a591a4af72577159 (DEFAULT_TABLE table=test-table [id=eb44b72694c44610905c8e7b0a0d262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.278471 4492 tablet_service.cc:1467] Processing CreateTablet for tablet 228df9ae0d2c4354a591a4af72577159 (DEFAULT_TABLE table=test-table [id=eb44b72694c44610905c8e7b0a0d262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.278501 4907 tablet_service.cc:1467] Processing CreateTablet for tablet 228df9ae0d2c4354a591a4af72577159 (DEFAULT_TABLE table=test-table [id=eb44b72694c44610905c8e7b0a0d262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:37.278673 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228df9ae0d2c4354a591a4af72577159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.278669 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228df9ae0d2c4354a591a4af72577159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.278733 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228df9ae0d2c4354a591a4af72577159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:37.280422 5173 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Bootstrap starting.
I20260807 08:29:37.280622 5168 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Bootstrap starting.
I20260807 08:29:37.280676 5171 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118: Bootstrap starting.
I20260807 08:29:37.281452 5168 tablet_bootstrap.cc:654] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.281615 5171 tablet_bootstrap.cc:654] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.282035 5173 tablet_bootstrap.cc:654] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:37.282604 5168 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: No bootstrap required, opened a new log
I20260807 08:29:37.282694 5168 ts_tablet_manager.cc:1397] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 08:29:37.283036 5171 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118: No bootstrap required, opened a new log
I20260807 08:29:37.283092 5171 ts_tablet_manager.cc:1397] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:37.283092 5168 raft_consensus.cc:348] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.283212 5168 raft_consensus.cc:374] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.283245 5168 raft_consensus.cc:729] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df2566b588439eac7dba2f1847a495, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.283358 5171 raft_consensus.cc:348] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.283381 5168 consensus_queue.cc:260] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.283453 5171 raft_consensus.cc:374] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.283479 5171 raft_consensus.cc:729] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c65555454a4912862b23ad7895a118, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.283569 5171 consensus_queue.cc:260] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.283643 5168 ts_tablet_manager.cc:1428] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:29:37.283818 5171 ts_tablet_manager.cc:1428] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:37.284027 5173 tablet_bootstrap.cc:492] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: No bootstrap required, opened a new log
I20260807 08:29:37.284097 5173 ts_tablet_manager.cc:1397] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 08:29:37.284505 5173 raft_consensus.cc:348] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.284591 5173 raft_consensus.cc:374] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:37.284611 5173 raft_consensus.cc:729] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Initialized, Role: FOLLOWER
I20260807 08:29:37.284705 5173 consensus_queue.cc:260] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.284938 5173 ts_tablet_manager.cc:1428] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:29:37.398252 5181 raft_consensus.cc:482] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:37.398458 5181 raft_consensus.cc:504] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.399663 5181 leader_election.cc:288] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737)
I20260807 08:29:37.403589 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.400113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:38161 (local address 127.1.157.66:33737)
0807 08:29:37.400246 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.400251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.400380 (+ 129us) server_negotiation.cc:407] Connection header received
0807 08:29:37.400511 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.400514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.400556 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.400613 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:37.400894 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.401539 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.402675 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.403084 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.403194 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.403261 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.403416 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.403490 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:29:37.403613 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.399917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.1.157.67:38161)
0807 08:29:37.400295 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:37.400306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:37.400405 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:37.400636 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:37.400639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:37.400654 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:37.400795 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.400801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.401632 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.401635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.402562 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.402572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.403194 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.403198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.403304 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.403325 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:29:37.403391 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:29:37.403456 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":18,"thread_start_us":110,"threads_started":1}
I20260807 08:29:37.403981 4726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228df9ae0d2c4354a591a4af72577159" candidate_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df2566b588439eac7dba2f1847a495" is_pre_election: true
I20260807 08:29:37.404120 4726 raft_consensus.cc:2454] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f2869ee0ae4b7cbb81f92e757221d0 in term 0.
I20260807 08:29:37.404306 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.399975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:59647 (local address 127.1.157.65:33139)
0807 08:29:37.400212 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.400225 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.400324 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:29:37.400457 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.400459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.400499 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.400569 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:37.401109 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.401880 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.403093 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.403551 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.403783 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.403842 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.403983 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.404052 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":66,"threads_started":1}
I20260807 08:29:37.404438 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.399906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33139 (local address 127.1.157.67:59647)
0807 08:29:37.400233 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:37.400247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:37.400359 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:37.400591 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:37.400594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:37.400610 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:37.400977 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.400988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.402041 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.402044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.402967 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.402978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.403670 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.403674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.403861 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.403884 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:37.403956 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:29:37.404021 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260807 08:29:37.404469 4831 leader_election.cc:302] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, d5f2869ee0ae4b7cbb81f92e757221d0; no voters:
I20260807 08:29:37.404706 5181 raft_consensus.cc:2781] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:37.404794 5181 raft_consensus.cc:482] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:37.404843 5181 raft_consensus.cc:3037] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.404830 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228df9ae0d2c4354a591a4af72577159" candidate_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118" is_pre_election: true
I20260807 08:29:37.404968 4518 raft_consensus.cc:2454] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f2869ee0ae4b7cbb81f92e757221d0 in term 0.
I20260807 08:29:37.406411 5181 raft_consensus.cc:504] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.406713 5181 leader_election.cc:288] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [CANDIDATE]: Term 1 election: Requested vote from peers 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139), 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737)
I20260807 08:29:37.407042 4726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228df9ae0d2c4354a591a4af72577159" candidate_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df2566b588439eac7dba2f1847a495"
I20260807 08:29:37.407145 4726 raft_consensus.cc:3037] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.408143 4726 raft_consensus.cc:2454] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f2869ee0ae4b7cbb81f92e757221d0 in term 1.
I20260807 08:29:37.408356 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228df9ae0d2c4354a591a4af72577159" candidate_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c65555454a4912862b23ad7895a118"
I20260807 08:29:37.408383 4831 leader_election.cc:302] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df2566b588439eac7dba2f1847a495, d5f2869ee0ae4b7cbb81f92e757221d0; no voters:
I20260807 08:29:37.408454 4518 raft_consensus.cc:3037] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:37.408550 5181 raft_consensus.cc:2781] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:37.408718 5181 raft_consensus.cc:686] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 LEADER]: Becoming Leader. State: Replica: d5f2869ee0ae4b7cbb81f92e757221d0, State: Running, Role: LEADER
I20260807 08:29:37.408962 5181 consensus_queue.cc:237] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } }
I20260807 08:29:37.409476 4518 raft_consensus.cc:2454] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f2869ee0ae4b7cbb81f92e757221d0 in term 1.
I20260807 08:29:37.411352 4289 catalog_manager.cc:5495] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67). New cstate: current_term: 1 leader_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5f2869ee0ae4b7cbb81f92e757221d0" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36413 } health_report { overall_health: HEALTHY } } }
I20260807 08:29:37.442153 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.438268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.1.157.67:36413)
0807 08:29:37.438612 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.438655 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.438667 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:37.438727 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.438729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.438776 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.438880 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:37.439108 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.439816 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.440588 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.440925 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.441017 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.441278 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.441726 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.441835 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.441925 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.441976 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":186,"threads_started":1}
I20260807 08:29:37.451673 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.448569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:59167 (local address 127.1.157.66:33737)
0807 08:29:37.448669 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.448672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.448930 (+ 258us) server_negotiation.cc:407] Connection header received
0807 08:29:37.449027 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.449029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.449072 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.449128 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:37.449367 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.449816 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.450726 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.450926 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.450993 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.451370 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.451427 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.451471 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.451538 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.451580 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:37.451797 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.448457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33737 (local address 127.1.157.67:59167)
0807 08:29:37.448864 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:37.448877 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:37.448960 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:37.449142 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:37.449144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:37.449148 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:37.449290 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.449296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.449885 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.449887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.450642 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.450652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.451100 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.451104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.451219 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.451272 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.451481 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.451485 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:29:37.451526 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 08:29:37.451581 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":136,"threads_started":1}
I20260807 08:29:37.456529 4726 raft_consensus.cc:1261] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Refusing update from remote peer d5f2869ee0ae4b7cbb81f92e757221d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:37.456532 4518 raft_consensus.cc:1261] T 228df9ae0d2c4354a591a4af72577159 P 72c65555454a4912862b23ad7895a118 [term 1 FOLLOWER]: Refusing update from remote peer d5f2869ee0ae4b7cbb81f92e757221d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:37.456997 5198 consensus_queue.cc:1035] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04df2566b588439eac7dba2f1847a495" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:37.457188 5181 consensus_queue.cc:1035] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72c65555454a4912862b23ad7895a118" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:37.472544 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.469131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:37289 (local address 127.1.157.67:36413)
0807 08:29:37.469393 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:37.469397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:37.469546 (+ 149us) server_negotiation.cc:407] Connection header received
0807 08:29:37.469725 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:37.469728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:37.469775 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:37.469841 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:37.470154 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.470656 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.471508 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.471757 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.471851 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.472170 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.472235 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.472296 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:37.472389 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 08:29:37.472434 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260807 08:29:37.472581 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:37.469049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36413 (local address 127.1.157.66:37289)
0807 08:29:37.469428 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:37.469441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:37.469619 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:37.469859 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:37.469861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:37.469867 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:29:37.470069 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.470075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.470733 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.470736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.471403 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:37.471415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:37.471873 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:37.471877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:37.472022 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:37.472055 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:29:37.472317 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:29:37.472323 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:29:37.472379 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:29:37.472460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":184,"threads_started":1}
W20260807 08:29:37.473294 5124 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 08:29:39.025956 4289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:39.026101 4289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:39.028139 4289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f94596a30c134f83858295e96f79702a: Sending DeleteTablet for 3 replicas of tablet 228df9ae0d2c4354a591a4af72577159
I20260807 08:29:39.028609 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet 228df9ae0d2c4354a591a4af72577159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:39 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:45932
I20260807 08:29:39.028609 4701 tablet_service.cc:1514] Processing DeleteTablet for tablet 228df9ae0d2c4354a591a4af72577159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:39 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37150
I20260807 08:29:39.028635 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet 228df9ae0d2c4354a591a4af72577159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:39 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33930
I20260807 08:29:39.028936 5250 tablet_replica.cc:330] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: stopping tablet replica
I20260807 08:29:39.029001 5253 tablet_replica.cc:330] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: stopping tablet replica
I20260807 08:29:39.029069 5250 raft_consensus.cc:2229] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:39.029125 5253 raft_consensus.cc:2229] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:39.029175 5250 raft_consensus.cc:2258] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:39.029238 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 4396
I20260807 08:29:39.034072 5253 raft_consensus.cc:2258] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:39.034771 5253 ts_tablet_manager.cc:1905] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:39.034914 5250 ts_tablet_manager.cc:1905] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:39.037279 5250 ts_tablet_manager.cc:1918] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:29:39.037335 5250 log.cc:1196] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/228df9ae0d2c4354a591a4af72577159
I20260807 08:29:39.037536 5250 ts_tablet_manager.cc:1939] T 228df9ae0d2c4354a591a4af72577159 P 04df2566b588439eac7dba2f1847a495: Deleting consensus metadata
I20260807 08:29:39.038126 4275 catalog_manager.cc:4841] TS 04df2566b588439eac7dba2f1847a495 (127.1.157.66:33737): tablet 228df9ae0d2c4354a591a4af72577159 (table test-table [id=eb44b72694c44610905c8e7b0a0d262a]) successfully deleted
I20260807 08:29:39.038218 5253 ts_tablet_manager.cc:1918] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 08:29:39.038316 5253 log.cc:1196] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/228df9ae0d2c4354a591a4af72577159
I20260807 08:29:39.038704 5253 ts_tablet_manager.cc:1939] T 228df9ae0d2c4354a591a4af72577159 P d5f2869ee0ae4b7cbb81f92e757221d0: Deleting consensus metadata
I20260807 08:29:39.039965 4271 catalog_manager.cc:4841] TS d5f2869ee0ae4b7cbb81f92e757221d0 (127.1.157.67:36413): tablet 228df9ae0d2c4354a591a4af72577159 (table test-table [id=eb44b72694c44610905c8e7b0a0d262a]) successfully deleted
W20260807 08:29:39.048107 4271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:33139 (error 108)
I20260807 08:29:39.048352 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 4606
I20260807 08:29:39.051877 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:39.051457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33139 (local address 127.0.0.1:57152)
0807 08:29:39.051693 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:39.051780 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
W20260807 08:29:39.052012 4271 catalog_manager.cc:4584] TS 72c65555454a4912862b23ad7895a118 (127.1.157.65:33139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 228df9ae0d2c4354a591a4af72577159: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33139: connect: Connection refused (error 111)
I20260807 08:29:39.056777 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 4808
I20260807 08:29:39.067312 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 4250
2026-08-07T08:29:39Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6379 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:39 dist-test-slave-6zbq krb5kdc[5259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:39 dist-test-slave-6zbq krb5kdc[5259](info): set up 2 sockets
Aug 07 08:29:39 dist-test-slave-6zbq krb5kdc[5259](info): commencing operation
krb5kdc: starting...
W20260807 08:29:40.191733 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:40.303046 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44935
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:40.408620 5288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:40.408838 5288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:40.408874 5288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:40.412707 5288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:40.412753 5288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:40.412772 5288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:40.412789 5288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:40.412817 5288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:40.417734 5288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:44935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:40.418603 5288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:40.419483 5288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:40.427314 5296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:40.427420 5288 server_base.cc:1034] running on GCE node
W20260807 08:29:40.427366 5299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:40.427376 5297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:40.428047 5288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:40.429009 5288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:40.430176 5288 hybrid_clock.cc:648] HybridClock initialized: now 1786091380430155 us; error 35 us; skew 500 ppm
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:40.432634 5288 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:40.433435 5288 webserver.cc:460] Webserver started at http://127.1.157.126:37663/ using document root <none> and password file <none>
I20260807 08:29:40.433923 5288 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:40.433988 5288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:40.434173 5288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:40.436107 5288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "abd21b641a5b49198bcc877b8e3a2662"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "69300a657b19a1f13c051131272eee22"
server_key_iv: "c911d98638e28740bd1a47e058d5c829"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.436625 5288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "abd21b641a5b49198bcc877b8e3a2662"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "69300a657b19a1f13c051131272eee22"
server_key_iv: "c911d98638e28740bd1a47e058d5c829"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.440131 5288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:40.442399 5315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.443545 5288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:40.443661 5288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "abd21b641a5b49198bcc877b8e3a2662"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "69300a657b19a1f13c051131272eee22"
server_key_iv: "c911d98638e28740bd1a47e058d5c829"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.443754 5288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:40.456643 5288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:40.459270 5288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:40.459430 5288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:40.466500 5288 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:44935
I20260807 08:29:40.466522 5399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:44935 every 8 connection(s)
I20260807 08:29:40.467360 5288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:40.468786 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 5288
I20260807 08:29:40.468899 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:40.469179 1653 external_mini_cluster.cc:1428] Setting key 431a204f51338bdb162f3b1b0d04c408
I20260807 08:29:40.470309 5400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:40.476297 5400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662: Bootstrap starting.
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:40.478622 5400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:40.479378 5400 log.cc:824] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:40.481043 5400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662: No bootstrap required, opened a new log
I20260807 08:29:40.482604 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:40.470466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.1.157.126:44935)
0807 08:29:40.470878 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:40.470893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:40.470926 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:29:40.471533 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:40.471548 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:40.471855 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:40.472108 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:40.472972 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.473925 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.475074 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.475523 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.475635 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:40.478096 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:40.478101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:40.478113 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:40.478139 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:40.479709 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.480102 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.480108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.480112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.480221 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.480477 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.480480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.480482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.480853 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:40.481035 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:40.481342 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 08:29:40.481628 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":127,"threads_started":1}
I20260807 08:29:40.485252 5400 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } }
I20260807 08:29:40.485446 5400 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:40.485486 5400 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd21b641a5b49198bcc877b8e3a2662, State: Initialized, Role: FOLLOWER
I20260807 08:29:40.485816 5400 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } }
I20260807 08:29:40.485924 5400 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:40.485965 5400 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:40.486030 5400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:40.486816 5400 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } }
I20260807 08:29:40.487082 5400 leader_election.cc:302] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd21b641a5b49198bcc877b8e3a2662; no voters:
I20260807 08:29:40.487308 5400 leader_election.cc:288] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:40.487414 5411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:40.487615 5411 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [term 1 LEADER]: Becoming Leader. State: Replica: abd21b641a5b49198bcc877b8e3a2662, State: Running, Role: LEADER
I20260807 08:29:40.487928 5411 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } }
I20260807 08:29:40.488315 5400 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:40.489362 5414 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd21b641a5b49198bcc877b8e3a2662. Latest consensus state: current_term: 1 leader_uuid: "abd21b641a5b49198bcc877b8e3a2662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } } }
I20260807 08:29:40.489490 5414 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:40.489777 5412 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd21b641a5b49198bcc877b8e3a2662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd21b641a5b49198bcc877b8e3a2662" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44935 } } }
I20260807 08:29:40.489861 5412 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:40.489863 5426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:40.492691 5426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:40.497706 5426 catalog_manager.cc:1347] Generated new cluster ID: 6d6bfe25af2744228f8dcfd2255a2ca2
I20260807 08:29:40.497768 5426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:40.510269 5426 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:40.511303 5426 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:40.520035 5426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abd21b641a5b49198bcc877b8e3a2662: Generated new TSK 0
I20260807 08:29:40.520560 5426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:40.544451 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:40.655124 5446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:40.655354 5446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:40.655398 5446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:40.659282 5446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:40.659327 5446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:40.659394 5446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:40.664355 5446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:40.665309 5446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:40.666375 5446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:40.675210 5455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:40.675335 5446 server_base.cc:1034] running on GCE node
W20260807 08:29:40.675215 5451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:40.675215 5452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:40.675848 5446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:40.676433 5446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:40.677680 5446 hybrid_clock.cc:648] HybridClock initialized: now 1786091380677643 us; error 47 us; skew 500 ppm
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:40.681740 5446 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:40.683743 5446 webserver.cc:460] Webserver started at http://127.1.157.65:36177/ using document root <none> and password file <none>
I20260807 08:29:40.684369 5446 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:40.684438 5446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:40.684626 5446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:40.686692 5446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "3847a05cf4c84b0aba3d2a8781f4d3aa"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "a0ac5a610000393bcd4228751dd4f4fa"
server_key_iv: "da3d94c917c4339f71b4b0308ab385d5"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.687250 5446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "3847a05cf4c84b0aba3d2a8781f4d3aa"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "a0ac5a610000393bcd4228751dd4f4fa"
server_key_iv: "da3d94c917c4339f71b4b0308ab385d5"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.691180 5446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:29:40.693459 5468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.694650 5446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:40.694747 5446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "3847a05cf4c84b0aba3d2a8781f4d3aa"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "a0ac5a610000393bcd4228751dd4f4fa"
server_key_iv: "da3d94c917c4339f71b4b0308ab385d5"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.694845 5446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:40.707866 5446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:40.710785 5446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:40.710966 5446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:40.711562 5446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:40.712298 5446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:40.712343 5446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.712379 5446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:40.712399 5446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.720443 5446 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:34957
I20260807 08:29:40.720521 5633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:34957 every 8 connection(s)
I20260807 08:29:40.721275 5446 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:40.730858 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 5446
I20260807 08:29:40.730962 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:40.731263 1653 external_mini_cluster.cc:1428] Setting key 8a86704b2a2a1311e768025f37feded0
I20260807 08:29:40.735378 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:40.722957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52511 (local address 127.1.157.126:44935)
0807 08:29:40.723077 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:40.723081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:40.723939 (+ 858us) server_negotiation.cc:407] Connection header received
0807 08:29:40.725572 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:40.725575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:40.725634 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:40.725706 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:40.726869 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.727418 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.728519 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.728765 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.728855 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:40.731459 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:40.731464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:40.731467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:40.731498 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:40.733236 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.733839 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.733843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.733846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.733942 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.734308 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.734310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.734311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.734511 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:40.734603 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:40.735086 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 08:29:40.735219 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:29:40.736168 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:40.723309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44935 (local address 127.1.157.65:52511)
0807 08:29:40.723800 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:40.723853 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:40.725346 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:40.725890 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:40.725896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:40.726225 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:40.726719 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:40.726733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.727541 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.727544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:40.728399 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:40.728409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.728905 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.728909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:40.729073 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:40.729677 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:40.729696 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:40.731284 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:40.733367 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:40.733374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:40.733388 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:40.733706 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:40.734036 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:40.734039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:40.734041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:40.734212 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:40.734623 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:40.734630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:40.734858 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 08:29:40.735024 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 08:29:40.735355 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260807 08:29:40.737268 5634 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44935
I20260807 08:29:40.737520 5634 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:40.738059 5634 heartbeater.cc:511] Master 127.1.157.126:44935 requested a full tablet report, sending...
I20260807 08:29:40.739619 5346 ts_manager.cc:194] Registered new tserver with Master: 3847a05cf4c84b0aba3d2a8781f4d3aa (127.1.157.65:34957)
I20260807 08:29:40.740713 5346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:52511
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:40.756868 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:40.863720 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:40.863938 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:40.863976 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:40.867787 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:40.867832 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:40.867897 5639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:40.872768 5639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:40.873610 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:40.874497 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:40.883009 5647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:40.883016 5651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:40.883015 5645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:40.883168 5639 server_base.cc:1034] running on GCE node
I20260807 08:29:40.883615 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:40.884191 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:40.885339 5639 hybrid_clock.cc:648] HybridClock initialized: now 1786091380885317 us; error 36 us; skew 500 ppm
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:40.887796 5639 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:40.888556 5639 webserver.cc:460] Webserver started at http://127.1.157.66:41521/ using document root <none> and password file <none>
I20260807 08:29:40.889075 5639 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:40.889158 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:40.889438 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:40.891316 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "2c50e2a9ea7d44f090fac20c94e73339"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "e325bfd539fd67bd13d0e4bde3463917"
server_key_iv: "7c0a2df1d95e3d9f105a26195701a34a"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.891829 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "2c50e2a9ea7d44f090fac20c94e73339"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "e325bfd539fd67bd13d0e4bde3463917"
server_key_iv: "7c0a2df1d95e3d9f105a26195701a34a"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.895121 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:40.897300 5663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.898386 5639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:40.898512 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "2c50e2a9ea7d44f090fac20c94e73339"
format_stamp: "Formatted at 2026-08-07 08:29:40 on dist-test-slave-6zbq"
server_key: "e325bfd539fd67bd13d0e4bde3463917"
server_key_iv: "7c0a2df1d95e3d9f105a26195701a34a"
server_key_version: "encryptionkey@0"
I20260807 08:29:40.898599 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:40.925604 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:40.928131 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:40.928292 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:40.928797 5639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:40.929513 5639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:40.929560 5639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.929610 5639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:40.929646 5639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:40.937860 5639 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:42303
I20260807 08:29:40.937885 5834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:42303 every 8 connection(s)
I20260807 08:29:40.938681 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:40.942458 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 5639
I20260807 08:29:40.942575 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:40.942832 1653 external_mini_cluster.cc:1428] Setting key c90f95ff13d74d9739face97c96c133d
Aug 07 08:29:40 dist-test-slave-6zbq krb5kdc[5259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091380, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:40.951597 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:40.940246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:38801 (local address 127.1.157.126:44935)
0807 08:29:40.940379 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:40.940382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:40.941111 (+ 729us) server_negotiation.cc:407] Connection header received
0807 08:29:40.942328 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:40.942331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:40.942410 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:40.942500 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:40.943630 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.944144 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.945218 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.945435 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.945522 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:40.947978 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:40.947982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:40.947985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:40.948013 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:40.949441 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.950101 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.950104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.950105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.950202 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:40.950593 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:40.950596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:40.950597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:40.950763 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:40.950836 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:40.951389 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 08:29:40.951485 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:29:40.952630 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:40.940552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44935 (local address 127.1.157.66:38801)
0807 08:29:40.940956 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:40.940996 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:40.942120 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:40.942669 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:40.942677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:40.943033 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:40.943485 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:40.943499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.944265 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.944268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:40.945093 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:40.945102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:40.945567 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:40.945571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:40.945750 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:40.946276 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:40.946295 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:40.947821 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:40.949654 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:40.949659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:40.949673 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:40.950005 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:40.950291 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:40.950294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:40.950296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:40.950508 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:40.950857 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:40.950864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:40.951156 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 08:29:40.951366 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 08:29:40.951689 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:40.953732 5835 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44935
I20260807 08:29:40.953933 5835 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:40.954378 5835 heartbeater.cc:511] Master 127.1.157.126:44935 requested a full tablet report, sending...
I20260807 08:29:40.955317 5346 ts_manager.cc:194] Registered new tserver with Master: 2c50e2a9ea7d44f090fac20c94e73339 (127.1.157.66:42303)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:29:40.955940 5346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:38801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:40.968842 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:41.074965 5841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:41.075191 5841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:41.075242 5841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:41.079115 5841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:41.079169 5841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:41.079234 5841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:41.085588 5841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:41.086823 5841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:41.088076 5841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:41.097447 5850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:41.097466 5849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:41.097502 5841 server_base.cc:1034] running on GCE node
W20260807 08:29:41.097752 5852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:41.098104 5841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:41.098740 5841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:41.099937 5841 hybrid_clock.cc:648] HybridClock initialized: now 1786091381099922 us; error 30 us; skew 500 ppm
Aug 07 08:29:41 dist-test-slave-6zbq krb5kdc[5259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091381, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:41.102494 5841 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:41.103307 5841 webserver.cc:460] Webserver started at http://127.1.157.67:35267/ using document root <none> and password file <none>
I20260807 08:29:41.103812 5841 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:41.103876 5841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:41.104059 5841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:41.105963 5841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "11d44f56db0647ec8a7fd434871a5244"
format_stamp: "Formatted at 2026-08-07 08:29:41 on dist-test-slave-6zbq"
server_key: "d377096c30e6bd03e6076793f3e3361c"
server_key_iv: "ddad1c539539817fe37da992082329fe"
server_key_version: "encryptionkey@0"
I20260807 08:29:41.106532 5841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "11d44f56db0647ec8a7fd434871a5244"
format_stamp: "Formatted at 2026-08-07 08:29:41 on dist-test-slave-6zbq"
server_key: "d377096c30e6bd03e6076793f3e3361c"
server_key_iv: "ddad1c539539817fe37da992082329fe"
server_key_version: "encryptionkey@0"
I20260807 08:29:41.109899 5841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:29:41.112138 5859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:41.113209 5841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:29:41.113312 5841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "11d44f56db0647ec8a7fd434871a5244"
format_stamp: "Formatted at 2026-08-07 08:29:41 on dist-test-slave-6zbq"
server_key: "d377096c30e6bd03e6076793f3e3361c"
server_key_iv: "ddad1c539539817fe37da992082329fe"
server_key_version: "encryptionkey@0"
I20260807 08:29:41.113400 5841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:41.126806 5841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:41.129235 5841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:41.129393 5841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:41.129908 5841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:41.130692 5841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:41.130740 5841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:41.130789 5841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:41.130818 5841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:41.139123 5841 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:44741
I20260807 08:29:41.139184 6016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:44741 every 8 connection(s)
I20260807 08:29:41.139931 5841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:41.148218 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 5841
I20260807 08:29:41.148313 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
Aug 07 08:29:41 dist-test-slave-6zbq krb5kdc[5259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091381, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:41.148574 1653 external_mini_cluster.cc:1428] Setting key f95d23461acc9729cc2d4db9d9c91c36
I20260807 08:29:41.152372 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:41.141517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:40189 (local address 127.1.157.126:44935)
0807 08:29:41.141675 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:41.141680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:41.142446 (+ 766us) server_negotiation.cc:407] Connection header received
0807 08:29:41.143694 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:41.143697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:41.143750 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:41.143827 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:41.144829 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:41.145350 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:41.146382 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:41.146627 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:41.146705 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:41.149018 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:41.149022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:41.149025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:41.149054 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:41.150562 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:41.151011 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:41.151014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:41.151016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:41.151095 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:41.151389 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:41.151393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:41.151394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:41.151566 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:41.151650 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:41.152094 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 08:29:41.152210 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:29:41.153017 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:41.141866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44935 (local address 127.1.157.67:40189)
0807 08:29:41.142296 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:41.142356 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:41.143476 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:41.143963 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:41.143969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:41.144287 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:41.144707 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:41.144719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:41.145434 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:41.145436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:41.146260 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:41.146269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:41.146768 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:41.146771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:41.146904 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:41.147423 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:41.147438 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:41.148867 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:41.150664 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:41.150669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:41.150682 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:41.150919 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:41.151173 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:41.151176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:41.151178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:41.151305 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:41.151660 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:41.151666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:41.151882 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 08:29:41.152034 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 08:29:41.152309 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":121,"threads_started":1}
I20260807 08:29:41.153964 6019 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44935
I20260807 08:29:41.154140 6019 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:41.154557 6019 heartbeater.cc:511] Master 127.1.157.126:44935 requested a full tablet report, sending...
I20260807 08:29:41.155447 5346 ts_manager.cc:194] Registered new tserver with Master: 11d44f56db0647ec8a7fd434871a5244 (127.1.157.67:44741)
I20260807 08:29:41.155949 5346 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:40189
I20260807 08:29:41.162168 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:29:41.176350 5404 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:41.175072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.1.157.126:44935)
0807 08:29:41.175189 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:41.175193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:41.175369 (+ 176us) server_negotiation.cc:407] Connection header received
0807 08:29:41.175486 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:41.175489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:41.175527 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:41.175587 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:41.176228 (+ 641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47364: BlockingRecv error: recv got EOF from 127.0.0.1:47364 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:29:41.178078 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 5446
I20260807 08:29:41.196830 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 5639
I20260807 08:29:41.212280 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 5841
I20260807 08:29:41.223253 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 5288
2026-08-07T08:29:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2152 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:41 dist-test-slave-6zbq krb5kdc[6037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:41 dist-test-slave-6zbq krb5kdc[6037](info): set up 2 sockets
Aug 07 08:29:41 dist-test-slave-6zbq krb5kdc[6037](info): commencing operation
krb5kdc: starting...
W20260807 08:29:42.368940 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:42.486176 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33313
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:42.592118 6058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:42.592357 6058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:42.592393 6058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:42.596217 6058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:42.596268 6058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:42.596287 6058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:42.596300 6058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:42.596313 6058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:42.601280 6058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:42.602160 6058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:42.603076 6058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:42.612573 6065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:42.612582 6064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:42.612584 6067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:42.612771 6058 server_base.cc:1034] running on GCE node
I20260807 08:29:42.613389 6058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:42.614744 6058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:42.615908 6058 hybrid_clock.cc:648] HybridClock initialized: now 1786091382615892 us; error 24 us; skew 500 ppm
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091382, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:42.622452 6058 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:42.623278 6058 webserver.cc:460] Webserver started at http://127.1.157.126:44885/ using document root <none> and password file <none>
I20260807 08:29:42.623799 6058 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:42.623865 6058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:42.624061 6058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:42.626025 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "8c6b0e54391448e29e9fef2e0a528498"
format_stamp: "Formatted at 2026-08-07 08:29:42 on dist-test-slave-6zbq"
server_key: "70e338b9a44b187dfa64d84560babf00"
server_key_iv: "117f8cdc31b784552b7c78c278880a64"
server_key_version: "encryptionkey@0"
I20260807 08:29:42.626623 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "8c6b0e54391448e29e9fef2e0a528498"
format_stamp: "Formatted at 2026-08-07 08:29:42 on dist-test-slave-6zbq"
server_key: "70e338b9a44b187dfa64d84560babf00"
server_key_iv: "117f8cdc31b784552b7c78c278880a64"
server_key_version: "encryptionkey@0"
I20260807 08:29:42.630016 6058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:29:42.632261 6076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:42.633385 6058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:42.633499 6058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "8c6b0e54391448e29e9fef2e0a528498"
format_stamp: "Formatted at 2026-08-07 08:29:42 on dist-test-slave-6zbq"
server_key: "70e338b9a44b187dfa64d84560babf00"
server_key_iv: "117f8cdc31b784552b7c78c278880a64"
server_key_version: "encryptionkey@0"
I20260807 08:29:42.633585 6058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:42.645251 6058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:42.647773 6058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:42.647933 6058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:42.654558 6058 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:33313
I20260807 08:29:42.654896 6149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:33313 every 8 connection(s)
I20260807 08:29:42.655431 6058 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:42.658013 6152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:42.661334 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6058
I20260807 08:29:42.661442 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:42.661677 1653 external_mini_cluster.cc:1428] Setting key 5ac912938e613257d04ef26f4a90952a
I20260807 08:29:42.663426 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: Bootstrap starting.
I20260807 08:29:42.665740 6152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:42.666492 6152 log.cc:824] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:42.668428 6152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: No bootstrap required, opened a new log
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:42.672114 6152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } }
I20260807 08:29:42.672329 6152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:42.672387 6152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c6b0e54391448e29e9fef2e0a528498, State: Initialized, Role: FOLLOWER
I20260807 08:29:42.672847 6152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } }
I20260807 08:29:42.673003 6152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:42.673072 6152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:42.673167 6152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:42.674165 6152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } }
I20260807 08:29:42.674520 6152 leader_election.cc:302] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c6b0e54391448e29e9fef2e0a528498; no voters:
I20260807 08:29:42.674867 6152 leader_election.cc:288] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:42.674952 6161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:42.675120 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:42.662850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.1.157.126:33313)
0807 08:29:42.663237 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:42.663252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:42.663286 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:29:42.663916 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:42.663935 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:42.664241 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:42.664497 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:42.665378 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:42.666378 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:42.667365 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:42.667813 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:42.667924 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:42.670553 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:42.670559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:42.670571 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:42.670605 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:42.672103 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:42.672527 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:42.672534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:42.672539 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:42.672678 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:42.672926 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:42.672929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:42.672930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:42.673450 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:42.673641 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:42.673861 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:29:42.674154 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":97,"threads_started":1}
I20260807 08:29:42.675144 6161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [term 1 LEADER]: Becoming Leader. State: Replica: 8c6b0e54391448e29e9fef2e0a528498, State: Running, Role: LEADER
I20260807 08:29:42.675644 6161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } }
I20260807 08:29:42.676014 6152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:42.678478 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:29:42.677726 6163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c6b0e54391448e29e9fef2e0a528498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } } }
I20260807 08:29:42.680998 6163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [sys.catalog]: This master's current role is: LEADER
W20260807 08:29:42.680312 6178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:29:42.681178 6178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:29:42.678211 6164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c6b0e54391448e29e9fef2e0a528498. Latest consensus state: current_term: 1 leader_uuid: "8c6b0e54391448e29e9fef2e0a528498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c6b0e54391448e29e9fef2e0a528498" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33313 } } }
I20260807 08:29:42.681289 6164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:42.681588 6184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:42.682518 6184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:42.687318 6184 catalog_manager.cc:1347] Generated new cluster ID: bd3c6a515f254a969cd7b30642cdc31e
I20260807 08:29:42.687417 6184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:42.693720 6184 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:42.694576 6184 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:42.701831 6184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c6b0e54391448e29e9fef2e0a528498: Generated new TSK 0
I20260807 08:29:42.702409 6184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:29:42.704046 6158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:42.698417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.1.157.126:33313)
0807 08:29:42.698522 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:42.698526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:42.698757 (+ 231us) server_negotiation.cc:407] Connection header received
0807 08:29:42.698923 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:42.698926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:42.698968 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:42.699016 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:42.699667 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:42.700547 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:42.701911 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:42.702283 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:42.702376 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:42.703315 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:42.703318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:42.703320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:42.703692 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:42.703719 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:42.703735 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:42.703806 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:42.703904 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:29:42.703949 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:29:42.708552 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6058
2026-08-07T08:29:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1488 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6197](info): set up 2 sockets
Aug 07 08:29:42 dist-test-slave-6zbq krb5kdc[6197](info): commencing operation
krb5kdc: starting...
W20260807 08:29:43.861737 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260807 08:29:44.026880 1653 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.165s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260807 08:29:44.282589 1653 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.256s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:44 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:44.422719 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44049
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:44049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:44.528457 6219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:44.528684 6219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:44.528738 6219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:44.532662 6219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:44.532716 6219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:44.532743 6219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:44.532764 6219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:44.532793 6219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:44.537762 6219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:44049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:44.538722 6219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:44.539656 6219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:44.547857 6230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:44.547942 6219 server_base.cc:1034] running on GCE node
W20260807 08:29:44.547866 6228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:44.547866 6235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:44.548658 6219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:44.549661 6219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:44.550836 6219 hybrid_clock.cc:648] HybridClock initialized: now 1786091384550818 us; error 26 us; skew 500 ppm
Aug 07 08:29:44 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091384, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:44.553360 6219 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:44.554172 6219 webserver.cc:460] Webserver started at http://127.1.157.126:46237/ using document root <none> and password file <none>
I20260807 08:29:44.554705 6219 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:44.554773 6219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:44.554971 6219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:44.556978 6219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "3ee32afb089a478d903f22fd17c60950"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "28807b0d941e83fba868d61aefae5cae"
server_key_iv: "de9d2f4cd88b1ce74d8ac5aa6d0d6352"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.557542 6219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "3ee32afb089a478d903f22fd17c60950"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "28807b0d941e83fba868d61aefae5cae"
server_key_iv: "de9d2f4cd88b1ce74d8ac5aa6d0d6352"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.561050 6219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:29:44.563234 6245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:44.564360 6219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:44.564472 6219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "3ee32afb089a478d903f22fd17c60950"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "28807b0d941e83fba868d61aefae5cae"
server_key_iv: "de9d2f4cd88b1ce74d8ac5aa6d0d6352"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.564558 6219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:44.579708 6219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:44.582182 6219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:44.582392 6219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:44.588814 6219 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:44049
I20260807 08:29:44.589048 6311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:44049 every 8 connection(s)
I20260807 08:29:44.589679 6219 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:44.592260 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:44.597415 6312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950: Bootstrap starting.
I20260807 08:29:44.597755 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6219
I20260807 08:29:44.597851 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:44.598089 1653 external_mini_cluster.cc:1428] Setting key 02aa5127be34a9d18242fc30c5847684
I20260807 08:29:44.599766 6312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:44.600454 6312 log.cc:824] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:44.602615 6312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950: No bootstrap required, opened a new log
I20260807 08:29:44.606566 6312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } }
I20260807 08:29:44.606851 6312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:44.606902 6312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee32afb089a478d903f22fd17c60950, State: Initialized, Role: FOLLOWER
I20260807 08:29:44.607372 6312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } }
I20260807 08:29:44.607527 6312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:44.607583 6312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:44.607683 6312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:44.608536 6312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } }
Aug 07 08:29:44 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:44.608902 6312 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee32afb089a478d903f22fd17c60950; no voters:
I20260807 08:29:44.609242 6312 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:44.609351 6320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:44.610186 6320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [term 1 LEADER]: Becoming Leader. State: Replica: 3ee32afb089a478d903f22fd17c60950, State: Running, Role: LEADER
I20260807 08:29:44.610572 6320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } }
I20260807 08:29:44.611231 6312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:44.611920 6320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee32afb089a478d903f22fd17c60950. Latest consensus state: current_term: 1 leader_uuid: "3ee32afb089a478d903f22fd17c60950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } } }
I20260807 08:29:44.612098 6320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:44.612316 6321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ee32afb089a478d903f22fd17c60950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee32afb089a478d903f22fd17c60950" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44049 } } }
I20260807 08:29:44.612422 6321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:44.612648 6324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:44.615739 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:44.599286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.1.157.126:44049)
0807 08:29:44.599670 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:44.599685 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:44.599720 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:29:44.600351 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:44.600368 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:44.600700 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:44.601050 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:44.602122 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.603590 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.604831 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.606492 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.606648 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:44.609707 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:44.609714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:44.609730 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:44.609763 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:44.611881 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:44.612412 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:44.612419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:44.612425 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:44.612572 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:44.612892 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:44.612895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:44.612897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:44.613442 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:44.614369 (+ 927us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:44.614534 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:29:44.614822 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260807 08:29:44.616024 6324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:44.626112 6324 catalog_manager.cc:1347] Generated new cluster ID: 92511c17bb864380812d139fe7d6afa5
I20260807 08:29:44.626185 6324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:44.638118 6324 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:44.639156 6324 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:44.645406 6324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee32afb089a478d903f22fd17c60950: Generated new TSK 0
I20260807 08:29:44.646116 6324 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:44.672817 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:44.779136 6351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:44.779369 6351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:44.779426 6351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:44.783313 6351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:44.783363 6351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:44.783437 6351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:44.788355 6351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:44.789260 6351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:44.790172 6351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:44.798943 6364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:44.798971 6360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:44.799038 6351 server_base.cc:1034] running on GCE node
W20260807 08:29:44.798985 6361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:44.799467 6351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:44.800038 6351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:44.801230 6351 hybrid_clock.cc:648] HybridClock initialized: now 1786091384801207 us; error 31 us; skew 500 ppm
Aug 07 08:29:44 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091384, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:44.803769 6351 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:44.804584 6351 webserver.cc:460] Webserver started at http://127.1.157.65:34079/ using document root <none> and password file <none>
I20260807 08:29:44.805078 6351 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:44.805150 6351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:44.805346 6351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:44.807279 6351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "f80565eb7b784d23ad5a0219f183c4f0"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "7ebbe1b26bdd2a7ce685837bec0c2524"
server_key_iv: "c27a63da8eb776be5073154e0b78c0a6"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.807794 6351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "f80565eb7b784d23ad5a0219f183c4f0"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "7ebbe1b26bdd2a7ce685837bec0c2524"
server_key_iv: "c27a63da8eb776be5073154e0b78c0a6"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.811167 6351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:44.813448 6378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:44.814646 6351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:44.814752 6351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "f80565eb7b784d23ad5a0219f183c4f0"
format_stamp: "Formatted at 2026-08-07 08:29:44 on dist-test-slave-6zbq"
server_key: "7ebbe1b26bdd2a7ce685837bec0c2524"
server_key_iv: "c27a63da8eb776be5073154e0b78c0a6"
server_key_version: "encryptionkey@0"
I20260807 08:29:44.814841 6351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:44.831579 6351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:44.834036 6351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:44.834206 6351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:44.834749 6351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:44.835501 6351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:44.835548 6351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:44.835597 6351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:44.835625 6351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:44.843902 6351 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:44615
I20260807 08:29:44.843938 6536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:44615 every 8 connection(s)
I20260807 08:29:44.844687 6351 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:44.848135 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6351
I20260807 08:29:44.848260 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:44.848551 1653 external_mini_cluster.cc:1428] Setting key 5491cb9841f70056ccafa951c6260f0e
Aug 07 08:29:44 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091384, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:44.858525 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:44.846269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52261 (local address 127.1.157.126:44049)
0807 08:29:44.846422 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:44.846426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:44.847193 (+ 767us) server_negotiation.cc:407] Connection header received
0807 08:29:44.848799 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:44.848802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:44.848853 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:44.848929 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:44.850198 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.850764 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.852265 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.852506 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.852598 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:44.854985 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:44.854989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:44.854991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:44.855019 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:44.856601 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:44.857065 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:44.857068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:44.857070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:44.857167 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:44.857456 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:44.857459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:44.857461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:44.857681 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:44.857794 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:44.858201 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 08:29:44.858332 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:29:44.859179 6542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:44.846640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44049 (local address 127.1.157.65:52261)
0807 08:29:44.847060 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:44.847099 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:44.848583 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:44.849085 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:44.849092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:44.849510 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:44.850055 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:44.850074 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.850891 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.850895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:44.852149 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:44.852160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:44.852645 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:44.852649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:44.852814 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:44.853328 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:44.853343 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:44.854823 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:44.856711 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:44.856716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:44.856726 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:44.856970 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:44.857250 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:44.857252 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:44.857253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:44.857373 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:44.857772 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:44.857776 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:44.857981 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 08:29:44.858129 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 08:29:44.858395 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:44.860317 6537 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44049
I20260807 08:29:44.860543 6537 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:44.860989 6537 heartbeater.cc:511] Master 127.1.157.126:44049 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:44.862568 6269 ts_manager.cc:194] Registered new tserver with Master: f80565eb7b784d23ad5a0219f183c4f0 (127.1.157.65:44615)
I20260807 08:29:44.863641 6269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:52261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:44.876881 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:44.983017 6545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:44.983265 6545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:44.983323 6545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:44.987268 6545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:44.987321 6545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:44.987398 6545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:44.992506 6545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:44.993549 6545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:44.994576 6545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:45.003860 6554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:45.003870 6558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:45.003870 6552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:45.004088 6545 server_base.cc:1034] running on GCE node
I20260807 08:29:45.004570 6545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:45.005198 6545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:45.006388 6545 hybrid_clock.cc:648] HybridClock initialized: now 1786091385006371 us; error 26 us; skew 500 ppm
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:45.008909 6545 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:45.009698 6545 webserver.cc:460] Webserver started at http://127.1.157.66:40397/ using document root <none> and password file <none>
I20260807 08:29:45.010205 6545 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:45.010269 6545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:45.010495 6545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:45.012542 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "5c830c2783ab44a38ee039396f75f0e1"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "cea7b1a655f311c216cfdf5b8459b4c4"
server_key_iv: "04a327b9346c195f245e144e9a94e530"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.013103 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "5c830c2783ab44a38ee039396f75f0e1"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "cea7b1a655f311c216cfdf5b8459b4c4"
server_key_iv: "04a327b9346c195f245e144e9a94e530"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.016623 6545 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:29:45.019138 6569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.020282 6545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:45.020394 6545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "5c830c2783ab44a38ee039396f75f0e1"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "cea7b1a655f311c216cfdf5b8459b4c4"
server_key_iv: "04a327b9346c195f245e144e9a94e530"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.020486 6545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:45.040177 6545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:45.042754 6545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:45.042917 6545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:45.043440 6545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:45.044201 6545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:45.044260 6545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.044304 6545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:45.044325 6545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.052397 6545 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:39825
I20260807 08:29:45.052445 6739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:39825 every 8 connection(s)
I20260807 08:29:45.053217 6545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:45.062740 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6545
I20260807 08:29:45.062868 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:45.063133 1653 external_mini_cluster.cc:1428] Setting key e48d9b8c7fd93be83ce5f571ae739eee
I20260807 08:29:45.065708 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:45.054824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46715 (local address 127.1.157.126:44049)
0807 08:29:45.054953 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:45.054957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:45.055745 (+ 788us) server_negotiation.cc:407] Connection header received
0807 08:29:45.057129 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:45.057133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:45.057188 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:45.057258 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:45.058412 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.058927 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.059996 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.060219 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.060308 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:45.062425 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:45.062429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:45.062431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:45.062456 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:45.063812 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.064267 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.064271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.064272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.064363 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.064725 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.064727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.064728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.064889 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:45.064961 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:45.065493 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 08:29:45.065592 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:29:45.066694 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:45.055143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44049 (local address 127.1.157.66:46715)
0807 08:29:45.055598 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:45.055645 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:45.056930 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:45.057422 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:45.057429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:45.057786 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:45.058250 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:45.058264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.059044 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.059046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:45.059881 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:45.059891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.060354 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.060358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:45.060506 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:45.060957 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:45.060972 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:45.062249 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:45.063916 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:45.063920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:45.063930 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:45.064170 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:45.064462 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:45.064465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:45.064466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:45.064639 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:45.064983 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:45.064989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:45.065277 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 08:29:45.065468 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 08:29:45.065760 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":116,"threads_started":1}
I20260807 08:29:45.067739 6740 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44049
I20260807 08:29:45.067958 6740 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:45.068390 6740 heartbeater.cc:511] Master 127.1.157.126:44049 requested a full tablet report, sending...
I20260807 08:29:45.069376 6269 ts_manager.cc:194] Registered new tserver with Master: 5c830c2783ab44a38ee039396f75f0e1 (127.1.157.66:39825)
I20260807 08:29:45.070111 6269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:46715
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:45.089520 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:45.194543 6746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:45.194749 6746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:45.194804 6746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:45.198609 6746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:45.198657 6746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:45.198730 6746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:45.203624 6746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:45.204452 6746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:45.205287 6746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:45.213779 6756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:45.213744 6753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:45.213764 6746 server_base.cc:1034] running on GCE node
W20260807 08:29:45.213796 6754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:45.214334 6746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:45.214906 6746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:45.216064 6746 hybrid_clock.cc:648] HybridClock initialized: now 1786091385216049 us; error 31 us; skew 500 ppm
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:45.218617 6746 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:45.219390 6746 webserver.cc:460] Webserver started at http://127.1.157.67:36395/ using document root <none> and password file <none>
I20260807 08:29:45.219859 6746 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:45.219923 6746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:45.220100 6746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:45.221895 6746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "abe6bb3be94644cea0a185af025c1327"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "e2fe5e0ed8806998e106239abf9bec7f"
server_key_iv: "a0bf2ab0f8fa1f30417dba92626d75c1"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.222432 6746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "abe6bb3be94644cea0a185af025c1327"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "e2fe5e0ed8806998e106239abf9bec7f"
server_key_iv: "a0bf2ab0f8fa1f30417dba92626d75c1"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.225708 6746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260807 08:29:45.227838 6768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.229705 6746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 08:29:45.229806 6746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "abe6bb3be94644cea0a185af025c1327"
format_stamp: "Formatted at 2026-08-07 08:29:45 on dist-test-slave-6zbq"
server_key: "e2fe5e0ed8806998e106239abf9bec7f"
server_key_iv: "a0bf2ab0f8fa1f30417dba92626d75c1"
server_key_version: "encryptionkey@0"
I20260807 08:29:45.229895 6746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:45.243546 6746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:45.245993 6746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:45.246155 6746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:45.246717 6746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:45.247470 6746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:45.247516 6746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.247568 6746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:45.247594 6746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:45.256191 6746 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:43497
I20260807 08:29:45.256287 6939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:43497 every 8 connection(s)
I20260807 08:29:45.257306 6746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:45.264969 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6746
I20260807 08:29:45.265097 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:45.265408 1653 external_mini_cluster.cc:1428] Setting key c8d47424f2aa43b2cb2c09b095b1c655
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:45.270735 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:45.259297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:53793 (local address 127.1.157.126:44049)
0807 08:29:45.259433 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:45.259437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:45.260298 (+ 861us) server_negotiation.cc:407] Connection header received
0807 08:29:45.261677 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:45.261680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:45.261731 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:45.261809 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:45.263089 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.263624 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.264765 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.265078 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.265191 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:45.267779 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:45.267783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:45.267785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:45.267814 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:45.269031 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.269485 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.269488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.269489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.269566 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.269818 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.269820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.269821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.269986 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:45.270056 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:45.270517 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 08:29:45.270612 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:29:45.271519 6942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:45.259660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44049 (local address 127.1.157.67:53793)
0807 08:29:45.260132 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:45.260177 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:45.261446 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:45.262017 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:45.262025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:45.262459 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:45.262938 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:45.262951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.263742 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.263745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:45.264616 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:45.264626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.265241 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.265244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:45.265385 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:45.265973 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:45.265989 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:45.267622 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:45.269131 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:45.269135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:45.269146 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:45.269411 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:45.269631 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:45.269633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:45.269634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:45.269755 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:45.270053 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:45.270058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:45.270288 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 08:29:45.270468 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 08:29:45.270711 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":128,"threads_started":1}
I20260807 08:29:45.272532 6940 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44049
I20260807 08:29:45.272742 6940 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:45.273186 6940 heartbeater.cc:511] Master 127.1.157.126:44049 requested a full tablet report, sending...
I20260807 08:29:45.274068 6269 ts_manager.cc:194] Registered new tserver with Master: abe6bb3be94644cea0a185af025c1327 (127.1.157.67:43497)
I20260807 08:29:45.274638 6269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:53793
I20260807 08:29:45.278867 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091385, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:45.303654 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:45.295902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.1.157.126:44049)
0807 08:29:45.296033 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:45.296038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:45.296205 (+ 167us) server_negotiation.cc:407] Connection header received
0807 08:29:45.296334 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:45.296338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:45.296379 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:45.296446 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:45.297158 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.297672 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.298537 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:45.298752 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:45.298858 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:45.300970 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:45.300973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:45.300975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:45.300999 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:45.302098 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.302682 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.302684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.302685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.302761 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:45.302954 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:45.302956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:45.302957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:45.303171 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:45.303271 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:45.303433 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 08:29:45.303524 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 08:29:45.304050 6269 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51108
I20260807 08:29:45.304898 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6351
I20260807 08:29:45.321002 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6545
I20260807 08:29:45.337519 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6746
I20260807 08:29:45.346661 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6219
2026-08-07T08:29:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2638 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6955](info): set up 2 sockets
Aug 07 08:29:45 dist-test-slave-6zbq krb5kdc[6955](info): commencing operation
krb5kdc: starting...
W20260807 08:29:46.469292 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:46 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:46.577472 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45809
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:45809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:46.690168 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:46.690431 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:46.690500 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:46.694442 6971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:46.694490 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:46.694515 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:46.694547 6971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:46.694576 6971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:46.699599 6971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:45809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:46.700474 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:46.701395 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:46.710827 6977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:46.710839 6978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:46.711256 6982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:46.712100 6971 server_base.cc:1034] running on GCE node
I20260807 08:29:46.712874 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:46.714267 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:46.715504 6971 hybrid_clock.cc:648] HybridClock initialized: now 1786091386715483 us; error 37 us; skew 500 ppm
Aug 07 08:29:46 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091386, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:46.718012 6971 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:46.718838 6971 webserver.cc:460] Webserver started at http://127.1.157.126:40419/ using document root <none> and password file <none>
I20260807 08:29:46.719326 6971 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:46.719383 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:46.719542 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:46.721460 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "803c66080b9a45209a26c7ad67271a67"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "45748ebf2b26dfdc82f62fbbdc9931d7"
server_key_iv: "66f50e3525420874063b3b5007f15bd5"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.721957 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "803c66080b9a45209a26c7ad67271a67"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "45748ebf2b26dfdc82f62fbbdc9931d7"
server_key_iv: "66f50e3525420874063b3b5007f15bd5"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.725421 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:29:46.727651 6995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:46.728760 6971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:46.728860 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "803c66080b9a45209a26c7ad67271a67"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "45748ebf2b26dfdc82f62fbbdc9931d7"
server_key_iv: "66f50e3525420874063b3b5007f15bd5"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.728952 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:46.751818 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:46.754292 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:46.754479 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:46.761034 6971 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:45809
I20260807 08:29:46.761070 7072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:45809 every 8 connection(s)
I20260807 08:29:46.761912 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:46.762490 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 6971
I20260807 08:29:46.762611 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:46.762910 1653 external_mini_cluster.cc:1428] Setting key 6f5ea495010cf5f6a8dc0591f6b31bfd
I20260807 08:29:46.764997 7073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:46.770128 7073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67: Bootstrap starting.
Aug 07 08:29:46 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:46.772570 7073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:46.773240 7073 log.cc:824] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:46.775235 7073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67: No bootstrap required, opened a new log
I20260807 08:29:46.775771 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:46.764239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.1.157.126:45809)
0807 08:29:46.764649 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:46.764663 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:46.764695 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:29:46.765333 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:46.765350 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:46.765654 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:46.765937 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:46.766720 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:46.767657 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:46.768473 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:46.768913 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:46.769050 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:46.771599 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:46.771604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:46.771615 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:46.771640 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:46.773155 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:46.773626 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:46.773632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:46.773638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:46.773788 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:46.774050 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:46.774053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:46.774054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:46.774444 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:46.774582 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:46.774849 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 08:29:46.775084 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":128,"threads_started":1}
I20260807 08:29:46.778326 7073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } }
I20260807 08:29:46.778550 7073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:46.778604 7073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803c66080b9a45209a26c7ad67271a67, State: Initialized, Role: FOLLOWER
I20260807 08:29:46.778980 7073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } }
I20260807 08:29:46.779103 7073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:46.779163 7073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:46.779253 7073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:46.779986 7073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } }
I20260807 08:29:46.780277 7073 leader_election.cc:302] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 803c66080b9a45209a26c7ad67271a67; no voters:
I20260807 08:29:46.780534 7073 leader_election.cc:288] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:46.780613 7079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:46.780826 7079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [term 1 LEADER]: Becoming Leader. State: Replica: 803c66080b9a45209a26c7ad67271a67, State: Running, Role: LEADER
I20260807 08:29:46.781163 7079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } }
I20260807 08:29:46.781513 7073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:46.782605 7081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 803c66080b9a45209a26c7ad67271a67. Latest consensus state: current_term: 1 leader_uuid: "803c66080b9a45209a26c7ad67271a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } } }
I20260807 08:29:46.782728 7081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:46.782979 7080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "803c66080b9a45209a26c7ad67271a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "803c66080b9a45209a26c7ad67271a67" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45809 } } }
I20260807 08:29:46.783157 7080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:46.783088 7091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:46.785648 7091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:46.790143 7091 catalog_manager.cc:1347] Generated new cluster ID: 741076772c2f4d86910761e377b3f4c6
I20260807 08:29:46.790208 7091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:46.797246 7091 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:46.798005 7091 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:46.806025 7091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 803c66080b9a45209a26c7ad67271a67: Generated new TSK 0
I20260807 08:29:46.806563 7091 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:46.838634 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:46.944181 7106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:46.944394 7106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:46.944432 7106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:46.948268 7106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:46.948313 7106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:46.948366 7106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:46.953187 7106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:46.954026 7106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:46.954945 7106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:46.963649 7111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:46.963641 7112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:46.963728 7106 server_base.cc:1034] running on GCE node
W20260807 08:29:46.963646 7114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:46.964313 7106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:46.964845 7106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:46.966005 7106 hybrid_clock.cc:648] HybridClock initialized: now 1786091386965987 us; error 27 us; skew 500 ppm
Aug 07 08:29:46 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091386, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:46.968652 7106 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:46.969437 7106 webserver.cc:460] Webserver started at http://127.1.157.65:45429/ using document root <none> and password file <none>
I20260807 08:29:46.969935 7106 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:46.969998 7106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:46.970191 7106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:46.972134 7106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "9a0366c85f274474a22a8a6af865ea47"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "fa6f8d051a4f00a341d217d86746dc87"
server_key_iv: "a05b6f8a4f530d494adef519f9ca416f"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.972730 7106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "9a0366c85f274474a22a8a6af865ea47"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "fa6f8d051a4f00a341d217d86746dc87"
server_key_iv: "a05b6f8a4f530d494adef519f9ca416f"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.976153 7106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 08:29:46.978324 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:46.979413 7106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:46.979514 7106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "9a0366c85f274474a22a8a6af865ea47"
format_stamp: "Formatted at 2026-08-07 08:29:46 on dist-test-slave-6zbq"
server_key: "fa6f8d051a4f00a341d217d86746dc87"
server_key_iv: "a05b6f8a4f530d494adef519f9ca416f"
server_key_version: "encryptionkey@0"
I20260807 08:29:46.979599 7106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:46.999640 7106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:47.002139 7106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:47.002305 7106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:47.002857 7106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:47.003626 7106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:47.003674 7106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.003722 7106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:47.003751 7106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.011729 7106 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:37689
I20260807 08:29:47.011777 7298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:37689 every 8 connection(s)
I20260807 08:29:47.012506 7106 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:47.013828 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 7106
I20260807 08:29:47.013940 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:47.014199 1653 external_mini_cluster.cc:1428] Setting key d045a72f30652a896bf83df24d6cf6ad
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091386, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:47.025960 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.014732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:54827 (local address 127.1.157.126:45809)
0807 08:29:47.014855 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:47.014859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:47.015216 (+ 357us) server_negotiation.cc:407] Connection header received
0807 08:29:47.016564 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:47.016567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:47.016621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:47.016693 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:47.017958 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.018529 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.020006 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.020224 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.020295 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.022794 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:47.022797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:47.022799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:47.022822 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:47.024126 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.024587 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.024590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.024591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.024667 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.024961 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.024964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.024966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.025145 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:47.025228 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:47.025680 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 08:29:47.025805 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:47.026719 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.014580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45809 (local address 127.1.157.65:54827)
0807 08:29:47.015088 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:47.015129 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:47.016378 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:47.016859 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:47.016865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:47.017189 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:47.017802 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.017820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.018660 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.018664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.019899 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.019910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.020466 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.020470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.020675 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.021261 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:47.021276 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:47.022646 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:47.024230 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.024236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.024250 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.024495 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.024747 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.024750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.024752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.024878 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.025243 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:47.025249 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:47.025462 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 08:29:47.025621 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 08:29:47.025908 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260807 08:29:47.027848 7301 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45809
I20260807 08:29:47.028092 7301 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:47.028612 7301 heartbeater.cc:511] Master 127.1.157.126:45809 requested a full tablet report, sending...
I20260807 08:29:47.030189 7023 ts_manager.cc:194] Registered new tserver with Master: 9a0366c85f274474a22a8a6af865ea47 (127.1.157.65:37689)
I20260807 08:29:47.031716 7023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:54827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:47.040532 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:47.145797 7308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:47.146000 7308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:47.146039 7308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:47.149859 7308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:47.149902 7308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:47.149955 7308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:47.154762 7308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:47.155580 7308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:47.156473 7308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:47.165210 7313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:47.165267 7314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:47.165207 7316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:47.165593 7308 server_base.cc:1034] running on GCE node
I20260807 08:29:47.165951 7308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:47.166553 7308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:47.167732 7308 hybrid_clock.cc:648] HybridClock initialized: now 1786091387167710 us; error 31 us; skew 500 ppm
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:47.170224 7308 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:47.171005 7308 webserver.cc:460] Webserver started at http://127.1.157.66:38523/ using document root <none> and password file <none>
I20260807 08:29:47.171471 7308 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:47.171526 7308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:47.171680 7308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:47.173480 7308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "653e97580f024fb6bcd143c974912052"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "2a15c9fb5067f76e239c66404e91c903"
server_key_iv: "d6cdf3d8dab9eb27dca540d620e9b744"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.173944 7308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "653e97580f024fb6bcd143c974912052"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "2a15c9fb5067f76e239c66404e91c903"
server_key_iv: "d6cdf3d8dab9eb27dca540d620e9b744"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.177277 7308 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 08:29:47.179442 7326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.180501 7308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:47.180591 7308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "653e97580f024fb6bcd143c974912052"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "2a15c9fb5067f76e239c66404e91c903"
server_key_iv: "d6cdf3d8dab9eb27dca540d620e9b744"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.180666 7308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:47.219924 7308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:47.222641 7308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:47.222790 7308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:47.223330 7308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:47.224089 7308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:47.224143 7308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.224181 7308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:47.224201 7308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.232507 7308 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:33813
I20260807 08:29:47.232539 7481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:33813 every 8 connection(s)
I20260807 08:29:47.233467 7308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:47.236119 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 7308
I20260807 08:29:47.236248 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:47.236567 1653 external_mini_cluster.cc:1428] Setting key 003fe3d17a4ddd4409b64c6a64bbe329
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:47.246666 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.235120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:43933 (local address 127.1.157.126:45809)
0807 08:29:47.235233 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:47.235236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:47.236063 (+ 827us) server_negotiation.cc:407] Connection header received
0807 08:29:47.237666 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:47.237669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:47.237718 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:47.237808 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:47.239238 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.239740 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.240914 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.241159 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.241247 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.243501 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:47.243505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:47.243507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:47.243533 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:47.244778 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.245231 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.245234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.245236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.245312 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.245613 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.245616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.245618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.245788 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:47.245872 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:47.246374 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 08:29:47.246496 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:47.247373 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.235455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45809 (local address 127.1.157.66:43933)
0807 08:29:47.235884 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:47.235939 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:47.237473 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:47.237975 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:47.237984 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:47.238487 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:47.239086 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.239115 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.239866 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.239870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.240797 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.240807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.241292 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.241297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.241446 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.241954 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:47.241969 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:47.243353 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:47.244879 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.244885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.244898 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.245142 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.245390 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.245393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.245395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.245521 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.245884 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:47.245891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:47.246122 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 08:29:47.246299 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 08:29:47.246608 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:29:47.248417 7483 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45809
I20260807 08:29:47.248646 7483 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:47.249090 7483 heartbeater.cc:511] Master 127.1.157.126:45809 requested a full tablet report, sending...
I20260807 08:29:47.250066 7023 ts_manager.cc:194] Registered new tserver with Master: 653e97580f024fb6bcd143c974912052 (127.1.157.66:33813)
I20260807 08:29:47.250629 7023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:43933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:47.261858 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:47.368290 7492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:47.368506 7492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:47.368563 7492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:47.372483 7492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:47.372534 7492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:47.372605 7492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:47.377830 7492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:47.378733 7492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:47.379614 7492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:47.388346 7505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:47.388368 7507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:47.388368 7503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:47.388401 7492 server_base.cc:1034] running on GCE node
I20260807 08:29:47.388911 7492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:47.389456 7492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:47.390610 7492 hybrid_clock.cc:648] HybridClock initialized: now 1786091387390592 us; error 32 us; skew 500 ppm
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:47.393129 7492 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:47.393893 7492 webserver.cc:460] Webserver started at http://127.1.157.67:41105/ using document root <none> and password file <none>
I20260807 08:29:47.394419 7492 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:47.394487 7492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:47.394677 7492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:47.396507 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "e6d54946ecf14c2cb38a27b4daab873d"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "e98a5759fa8bfe76d7cbf6cd9fdd1302"
server_key_iv: "b8eb8bc221fd916e24e7176ce3ebfcd8"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.397037 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "e6d54946ecf14c2cb38a27b4daab873d"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "e98a5759fa8bfe76d7cbf6cd9fdd1302"
server_key_iv: "b8eb8bc221fd916e24e7176ce3ebfcd8"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.400487 7492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:47.402648 7515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.403710 7492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:47.403813 7492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "e6d54946ecf14c2cb38a27b4daab873d"
format_stamp: "Formatted at 2026-08-07 08:29:47 on dist-test-slave-6zbq"
server_key: "e98a5759fa8bfe76d7cbf6cd9fdd1302"
server_key_iv: "b8eb8bc221fd916e24e7176ce3ebfcd8"
server_key_version: "encryptionkey@0"
I20260807 08:29:47.403898 7492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:47.421203 7492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:47.424103 7492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:47.424266 7492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:47.424758 7492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:47.425483 7492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:47.425529 7492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.425577 7492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:47.425606 7492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:47.433769 7492 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39457
I20260807 08:29:47.433796 7679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39457 every 8 connection(s)
I20260807 08:29:47.434546 7492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:47.437400 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 7492
I20260807 08:29:47.437525 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:47.437798 1653 external_mini_cluster.cc:1428] Setting key c3a07d73d0a1d45cfde1dce7b5f73928
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:47.446556 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.436054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:42363 (local address 127.1.157.126:45809)
0807 08:29:47.436181 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:47.436185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:47.436929 (+ 744us) server_negotiation.cc:407] Connection header received
0807 08:29:47.438568 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:47.438571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:47.438634 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:47.438711 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:47.439936 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.440456 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.441510 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.441740 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.441820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.443909 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:47.443912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:47.443914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:47.443938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:47.445005 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.445415 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.445417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.445418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.445481 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.445727 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.445729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.445730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.445879 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:47.445947 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:47.446318 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 08:29:47.446444 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:29:47.447219 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.436377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45809 (local address 127.1.157.67:42363)
0807 08:29:47.436791 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:47.436837 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:47.438369 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:47.438908 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:47.438917 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:47.439301 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:47.439791 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.439811 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.440553 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.440556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.441390 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:47.441399 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.441850 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.441852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:47.441977 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.442469 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:47.442484 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:47.443765 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:47.445095 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.445100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.445109 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.445344 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.445541 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:47.445544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:47.445545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:47.445665 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:47.445946 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:47.445950 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:47.446138 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 08:29:47.446286 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 08:29:47.446528 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":112,"threads_started":1}
I20260807 08:29:47.448130 7681 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45809
I20260807 08:29:47.448307 7681 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:47.448685 7681 heartbeater.cc:511] Master 127.1.157.126:45809 requested a full tablet report, sending...
I20260807 08:29:47.449542 7023 ts_manager.cc:194] Registered new tserver with Master: e6d54946ecf14c2cb38a27b4daab873d (127.1.157.67:39457)
I20260807 08:29:47.450057 7023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:42363
I20260807 08:29:47.451292 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:47.475203 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.467112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.1.157.65:37689)
0807 08:29:47.467417 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:47.467427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:47.467447 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:29:47.467492 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:47.467496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:47.467646 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:47.467750 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:47.468322 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.469058 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.469807 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.470157 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.470234 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.472215 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:47.472224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:47.472235 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:47.472259 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:47.473761 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.474070 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.474074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.474075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.474165 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.474366 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.474369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.474371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.474588 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:47.474747 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:47.474879 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 08:29:47.475009 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":112,"threads_started":1}
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[6955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:47.484709 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:47.477481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.1.157.126:45809)
0807 08:29:47.477598 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:47.477603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:47.477667 (+ 64us) server_negotiation.cc:407] Connection header received
0807 08:29:47.477777 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:47.477780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:47.477820 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:47.477882 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:47.478528 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.479013 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.479722 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:47.479936 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:47.480017 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:47.482004 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:47.482007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:47.482009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:47.482030 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:47.483263 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.483681 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.483684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.483685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.483756 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:47.483988 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:47.483990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:47.483991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:47.484180 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:47.484286 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:47.484475 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 08:29:47.484590 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 08:29:47.485026 7023 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47092
I20260807 08:29:47.485500 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 7106
I20260807 08:29:47.499646 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 7308
I20260807 08:29:47.512217 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 7492
I20260807 08:29:47.521482 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 6971
2026-08-07T08:29:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2172 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[7703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[7703](info): set up 2 sockets
Aug 07 08:29:47 dist-test-slave-6zbq krb5kdc[7703](info): commencing operation
krb5kdc: starting...
W20260807 08:29:48.642961 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:48 dist-test-slave-6zbq krb5kdc[7703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:48.750254 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39611
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:48.853438 7721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:48.853648 7721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:48.853685 7721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:48.857553 7721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:48.857602 7721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:48.857620 7721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:48.857635 7721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:48.857651 7721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:48.862530 7721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:48.863349 7721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:48.864180 7721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:48.871592 7740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:48.871591 7738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:48.871614 7737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:48.871898 7721 server_base.cc:1034] running on GCE node
I20260807 08:29:48.872377 7721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:48.873281 7721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:48.874398 7721 hybrid_clock.cc:648] HybridClock initialized: now 1786091388874378 us; error 30 us; skew 500 ppm
Aug 07 08:29:48 dist-test-slave-6zbq krb5kdc[7703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091388, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:48.879431 7721 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:48.880100 7721 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:48.880177 7721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:48.880411 7721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:48.882238 7721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "c17dd0b1f23d495b9ab99e0099f194a8"
format_stamp: "Formatted at 2026-08-07 08:29:48 on dist-test-slave-6zbq"
server_key: "1767bcfea9375c30ac3a942e8e48c973"
server_key_iv: "76e7001f10038988e2b382facfe38c52"
server_key_version: "encryptionkey@0"
I20260807 08:29:48.882789 7721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "c17dd0b1f23d495b9ab99e0099f194a8"
format_stamp: "Formatted at 2026-08-07 08:29:48 on dist-test-slave-6zbq"
server_key: "1767bcfea9375c30ac3a942e8e48c973"
server_key_iv: "76e7001f10038988e2b382facfe38c52"
server_key_version: "encryptionkey@0"
I20260807 08:29:48.885999 7721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:29:48.888129 7748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:48.889187 7721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:48.889298 7721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c17dd0b1f23d495b9ab99e0099f194a8"
format_stamp: "Formatted at 2026-08-07 08:29:48 on dist-test-slave-6zbq"
server_key: "1767bcfea9375c30ac3a942e8e48c973"
server_key_iv: "76e7001f10038988e2b382facfe38c52"
server_key_version: "encryptionkey@0"
I20260807 08:29:48.889384 7721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:48.903739 7721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:48.906062 7721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:48.906224 7721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:48.912581 7721 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:39611
I20260807 08:29:48.912807 7816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:39611 every 8 connection(s)
I20260807 08:29:48.913129 7721 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:48.914817 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 7721
I20260807 08:29:48.914937 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:48.915215 1653 external_mini_cluster.cc:1428] Setting key 3d4d96d4831d761a8610be04a462e359
I20260807 08:29:48.915949 7817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:48.921854 7817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: Bootstrap starting.
Aug 07 08:29:48 dist-test-slave-6zbq krb5kdc[7703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:48.924453 7817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:48.925108 7817 log.cc:824] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:48.927021 7817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: No bootstrap required, opened a new log
I20260807 08:29:48.928278 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:48.916416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57010 (local address 127.1.157.126:39611)
0807 08:29:48.916783 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:48.916798 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:48.916830 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:29:48.917425 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:48.917440 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:48.917721 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:48.918008 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:48.918843 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:48.919775 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:48.920921 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:48.921363 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:48.921466 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:48.924253 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:48.924258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:48.924270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:48.924296 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:48.925805 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:48.926233 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:48.926237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:48.926240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:48.926367 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:48.926620 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:48.926622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:48.926623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:48.926960 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:48.927100 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:48.927381 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 08:29:48.927621 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
I20260807 08:29:48.930653 7817 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } }
I20260807 08:29:48.930845 7817 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:48.930896 7817 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17dd0b1f23d495b9ab99e0099f194a8, State: Initialized, Role: FOLLOWER
I20260807 08:29:48.931278 7817 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } }
I20260807 08:29:48.931401 7817 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:48.931461 7817 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:48.931545 7817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:48.932296 7817 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } }
I20260807 08:29:48.932585 7817 leader_election.cc:302] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17dd0b1f23d495b9ab99e0099f194a8; no voters:
I20260807 08:29:48.932843 7817 leader_election.cc:288] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:48.932929 7822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:48.933132 7822 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [term 1 LEADER]: Becoming Leader. State: Replica: c17dd0b1f23d495b9ab99e0099f194a8, State: Running, Role: LEADER
I20260807 08:29:48.933454 7822 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } }
I20260807 08:29:48.933905 7817 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:48.934988 7823 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } } }
I20260807 08:29:48.935115 7823 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:48.935488 7834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:48.935451 7825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17dd0b1f23d495b9ab99e0099f194a8. Latest consensus state: current_term: 1 leader_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17dd0b1f23d495b9ab99e0099f194a8" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39611 } } }
I20260807 08:29:48.935595 7825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:48.938498 7834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:48.943202 7834 catalog_manager.cc:1347] Generated new cluster ID: 1107400e371c40e1bcedc6fe4db2a4c3
I20260807 08:29:48.943264 7834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:48.951125 7834 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:48.951902 7834 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:48.967712 7834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: Generated new TSK 0
I20260807 08:29:48.968302 7834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:49.002626 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:49.107869 7851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:49.108085 7851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:49.108130 7851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:49.112033 7851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:49.112097 7851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:49.112174 7851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:49.117987 7851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:49.118966 7851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:49.119900 7851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:49.128443 7863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:49.128464 7861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:49.128532 7866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:49.128643 7851 server_base.cc:1034] running on GCE node
I20260807 08:29:49.129113 7851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:49.129668 7851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:49.130806 7851 hybrid_clock.cc:648] HybridClock initialized: now 1786091389130784 us; error 36 us; skew 500 ppm
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:49.133347 7851 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:49.133955 7851 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:49.134013 7851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:49.134192 7851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:49.136116 7851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "47418e13027b45f0b832f964614d6b21"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "9a43128323a7b3e97362112704788ee6"
server_key_iv: "1a958c4ea075bb5ba689c7510373b85a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.136607 7851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "47418e13027b45f0b832f964614d6b21"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "9a43128323a7b3e97362112704788ee6"
server_key_iv: "1a958c4ea075bb5ba689c7510373b85a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.140044 7851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:29:49.142131 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.143263 7851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:49.143354 7851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "47418e13027b45f0b832f964614d6b21"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "9a43128323a7b3e97362112704788ee6"
server_key_iv: "1a958c4ea075bb5ba689c7510373b85a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.143428 7851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:49.156286 7851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:49.158954 7851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:49.159085 7851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:49.159554 7851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:49.160269 7851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:49.160313 7851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.160348 7851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:49.160367 7851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.168396 7851 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:40839
I20260807 08:29:49.168437 8031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:40839 every 8 connection(s)
I20260807 08:29:49.168893 7851 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:49.178504 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 7851
I20260807 08:29:49.178635 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:49.178925 1653 external_mini_cluster.cc:1428] Setting key b06938a9098d99c359483b0d2e52a4cc
I20260807 08:29:49.181391 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.170406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:36073 (local address 127.1.157.126:39611)
0807 08:29:49.170514 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.170517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.171280 (+ 763us) server_negotiation.cc:407] Connection header received
0807 08:29:49.172539 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.172542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.172601 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.172682 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:49.173788 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.174322 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.175431 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.175672 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.175763 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.177899 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:49.177903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:49.177905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:49.177930 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:49.179386 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.179884 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.179887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.179889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.179987 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.180344 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.180346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.180347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.180526 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:49.180602 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.181170 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.181275 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:29:49.182363 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.170700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39611 (local address 127.1.157.65:36073)
0807 08:29:49.171139 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.171179 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.172331 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.172855 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.172862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.173198 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:49.173630 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.173646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.174467 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.174470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.175310 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.175320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.175809 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.175814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.175966 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.176429 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:49.176444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:49.177746 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:49.179499 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.179506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.179518 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.179784 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.180076 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.180079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.180082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.180239 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.180628 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:49.180635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:49.180914 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.181131 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.181442 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":106,"threads_started":1}
I20260807 08:29:49.183609 8032 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39611
I20260807 08:29:49.183825 8032 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:49.184226 8032 heartbeater.cc:511] Master 127.1.157.126:39611 requested a full tablet report, sending...
I20260807 08:29:49.185833 7767 ts_manager.cc:194] Registered new tserver with Master: 47418e13027b45f0b832f964614d6b21 (127.1.157.65:40839)
I20260807 08:29:49.186959 7767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:36073
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:49.204787 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:49.309461 8037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:49.309695 8037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:49.309754 8037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:49.313633 8037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:49.313686 8037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:49.313764 8037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:49.318954 8037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:49.319844 8037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:49.320763 8037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:49.329373 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:49.329380 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:49.329526 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:49.329773 8037 server_base.cc:1034] running on GCE node
I20260807 08:29:49.330123 8037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:49.330691 8037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:49.331861 8037 hybrid_clock.cc:648] HybridClock initialized: now 1786091389331841 us; error 28 us; skew 500 ppm
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:49.334556 8037 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:49.335218 8037 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:49.335279 8037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:49.335426 8037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:49.337287 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "8176d1a69f094511ab13f9c0fbc9e340"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "7619627f0d91f5023f3883d6662e084a"
server_key_iv: "7a174012a28d9b0dbbe93b0714d1ae8a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.337854 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "8176d1a69f094511ab13f9c0fbc9e340"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "7619627f0d91f5023f3883d6662e084a"
server_key_iv: "7a174012a28d9b0dbbe93b0714d1ae8a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.341214 8037 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260807 08:29:49.343348 8058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.344437 8037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:49.344532 8037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "8176d1a69f094511ab13f9c0fbc9e340"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "7619627f0d91f5023f3883d6662e084a"
server_key_iv: "7a174012a28d9b0dbbe93b0714d1ae8a"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.344624 8037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:49.353705 8037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:49.356444 8037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:49.356595 8037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:49.357107 8037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:49.357826 8037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:49.357872 8037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.357909 8037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:49.357939 8037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.365933 8037 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:34625
I20260807 08:29:49.365995 8211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:34625 every 8 connection(s)
I20260807 08:29:49.366465 8037 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:49.369879 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8037
I20260807 08:29:49.369997 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:49.370271 1653 external_mini_cluster.cc:1428] Setting key 5c33485527bbdf281512a9fc4c042260
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:49.378475 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.367972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:53839 (local address 127.1.157.126:39611)
0807 08:29:49.368082 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.368085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.369021 (+ 936us) server_negotiation.cc:407] Connection header received
0807 08:29:49.369841 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.369845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.369902 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.369978 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:49.371196 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.371808 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.372875 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.373095 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.373190 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.375363 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:49.375366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:49.375369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:49.375393 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:49.376622 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.377072 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.377076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.377078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.377160 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.377455 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.377458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.377460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.377629 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:49.377715 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.378170 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.378295 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:29:49.379158 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.368259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39611 (local address 127.1.157.66:53839)
0807 08:29:49.368881 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.368917 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.369630 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.370141 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.370149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.370530 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:49.371047 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.371061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.371928 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.371931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.372757 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.372766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.373224 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.373228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.373382 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.373858 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:49.373873 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:49.375212 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:49.376728 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.376734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.376747 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.376982 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.377240 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.377243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.377245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.377370 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.377727 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:49.377733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:49.377947 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.378101 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.378414 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:49.380154 8212 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39611
I20260807 08:29:49.380369 8212 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:49.380777 8212 heartbeater.cc:511] Master 127.1.157.126:39611 requested a full tablet report, sending...
I20260807 08:29:49.381764 7767 ts_manager.cc:194] Registered new tserver with Master: 8176d1a69f094511ab13f9c0fbc9e340 (127.1.157.66:34625)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:29:49.382507 7767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:53839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:49.394585 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:49.500221 8218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:49.500440 8218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:49.500487 8218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:49.504364 8218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:49.504412 8218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:49.504484 8218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:49.509359 8218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:49.510214 8218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:49.511263 8218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:49.519930 8227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:49.520023 8218 server_base.cc:1034] running on GCE node
W20260807 08:29:49.520123 8224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:49.519949 8223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:49.520519 8218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:49.521044 8218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:49.522212 8218 hybrid_clock.cc:648] HybridClock initialized: now 1786091389522200 us; error 29 us; skew 500 ppm
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:49.524751 8218 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:49.525425 8218 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:49.525489 8218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:49.525668 8218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:49.527686 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "b3377e7c63764016af50965dccb3c403"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "3ce7459e82c25387e5197d7a5883e6ec"
server_key_iv: "94b3e05fd48d21796325a9f04fe505f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.528160 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "b3377e7c63764016af50965dccb3c403"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "3ce7459e82c25387e5197d7a5883e6ec"
server_key_iv: "94b3e05fd48d21796325a9f04fe505f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.531411 8218 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:49.533449 8234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.534606 8218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:49.534693 8218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "b3377e7c63764016af50965dccb3c403"
format_stamp: "Formatted at 2026-08-07 08:29:49 on dist-test-slave-6zbq"
server_key: "3ce7459e82c25387e5197d7a5883e6ec"
server_key_iv: "94b3e05fd48d21796325a9f04fe505f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:49.534770 8218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:49.557338 8218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:49.560303 8218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:49.560451 8218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:49.561019 8218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:49.561810 8218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:49.561863 8218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.561905 8218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:49.561925 8218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:49.570014 8218 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39653
I20260807 08:29:49.570042 8400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39653 every 8 connection(s)
I20260807 08:29:49.570576 8218 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:29:49 dist-test-slave-6zbq krb5kdc[7703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091389, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:49.579909 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8218
I20260807 08:29:49.580014 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:49.580339 1653 external_mini_cluster.cc:1428] Setting key 16cd6fb4a8e879adcf33575072a9ccc6
I20260807 08:29:49.583051 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.572214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:37039 (local address 127.1.157.126:39611)
0807 08:29:49.572353 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.572356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.573117 (+ 761us) server_negotiation.cc:407] Connection header received
0807 08:29:49.574411 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.574414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.574470 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.574546 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:49.575642 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.576170 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.577253 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.577476 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.577562 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.579731 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:49.579734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:49.579736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:49.579768 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:49.581098 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.581638 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.581641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.581642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.581721 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.582022 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.582025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.582027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.582204 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:49.582287 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.582779 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.582896 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:29:49.583737 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.572546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39611 (local address 127.1.157.67:37039)
0807 08:29:49.572975 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.573016 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.574178 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.574716 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.574723 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.575043 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:49.575495 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.575509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.576292 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.576295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.577126 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.577136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.577608 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.577612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.577769 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.578250 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:49.578273 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:49.579579 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:49.581201 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.581206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.581219 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.581533 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.581801 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:49.581804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:49.581806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:49.581936 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:49.582302 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:49.582308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:49.582559 (+ 251us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.582716 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.583014 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260807 08:29:49.584630 8401 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39611
I20260807 08:29:49.584841 8401 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:49.585230 8401 heartbeater.cc:511] Master 127.1.157.126:39611 requested a full tablet report, sending...
I20260807 08:29:49.586107 7767 ts_manager.cc:194] Registered new tserver with Master: b3377e7c63764016af50965dccb3c403 (127.1.157.67:39653)
I20260807 08:29:49.586668 7767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:37039
I20260807 08:29:49.593590 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:29:49.601178 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.595313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.1.157.126:39611)
0807 08:29:49.595439 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.595443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.595557 (+ 114us) server_negotiation.cc:407] Connection header received
0807 08:29:49.595662 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.595664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.595699 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.595750 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:49.596339 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.596805 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.597525 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.597715 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.597781 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.598909 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:49.598911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:49.598913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:49.598930 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:49.599931 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.600264 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.600266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.600267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.600328 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:49.600506 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:49.600508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:49.600509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:49.600688 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:49.600784 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.600937 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.601016 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:29:49.603628 7767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:49.606446 7767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:49.619179 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.613990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39653 (local address 127.0.0.1:60042)
0807 08:29:49.614714 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.614726 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.614814 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.615430 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.615433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.615449 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:49.615535 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.615541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616529 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.616532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.617692 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.617702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618685 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.618796 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.618837 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.618935 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.619046 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":91,"thread_start_us":99,"threads_started":1}
I20260807 08:29:49.619694 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.614705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.1.157.65:40839)
0807 08:29:49.615112 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.615122 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.615143 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:29:49.615213 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.615217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.615384 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.615534 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:49.615683 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616742 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618077 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618679 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618795 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.618932 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.619447 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.619545 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260807 08:29:49.619688 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.614244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.1.157.67:39653)
0807 08:29:49.614581 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.614592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.614907 (+ 315us) server_negotiation.cc:407] Connection header received
0807 08:29:49.614985 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.614990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.615177 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.615319 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:49.615666 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616387 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.617803 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618557 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618667 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.618795 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.619463 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.619545 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":150,"threads_started":1}
I20260807 08:29:49.619866 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.613774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.1.157.66:34625)
0807 08:29:49.614405 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.614417 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.614698 (+ 281us) server_negotiation.cc:407] Connection header received
0807 08:29:49.614791 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.614796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.615000 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.615152 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:49.615412 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616566 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618029 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618835 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618931 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.619558 (+ 627us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.619617 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.619723 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":135,"threads_started":1}
I20260807 08:29:49.619369 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.613620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:34625 (local address 127.0.0.1:37610)
0807 08:29:49.614394 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.614442 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.614565 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.615158 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.615163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.615182 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:49.615306 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.615313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616713 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.616716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.617903 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.617915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618946 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.619038 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.619061 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.619122 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.619209 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":74,"threads_started":1}
I20260807 08:29:49.619375 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.614276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:40839 (local address 127.0.0.1:55074)
0807 08:29:49.614893 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.614905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.614973 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.615519 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.615522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.615535 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:49.615606 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.615611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.616875 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.616879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.617986 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.617994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.618992 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.618996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.619078 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.619101 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.619165 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.619241 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260807 08:29:49.622419 8121 tablet_service.cc:1467] Processing CreateTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e (DEFAULT_TABLE table=test-table [id=0c22e19b7e1840c79e2487586fb33c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:49.622583 8308 tablet_service.cc:1467] Processing CreateTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e (DEFAULT_TABLE table=test-table [id=0c22e19b7e1840c79e2487586fb33c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:49.623008 7938 tablet_service.cc:1467] Processing CreateTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e (DEFAULT_TABLE table=test-table [id=0c22e19b7e1840c79e2487586fb33c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:49.623461 8121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48738f0c9bcb4cd48b3797dd96a5d92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:49.623588 8308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48738f0c9bcb4cd48b3797dd96a5d92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:49.623884 7938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48738f0c9bcb4cd48b3797dd96a5d92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:49.629058 8432 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Bootstrap starting.
I20260807 08:29:49.630174 8433 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Bootstrap starting.
I20260807 08:29:49.631073 8434 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Bootstrap starting.
I20260807 08:29:49.631884 8432 tablet_bootstrap.cc:654] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:49.632267 8433 tablet_bootstrap.cc:654] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:49.633141 8434 tablet_bootstrap.cc:654] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:49.633190 8432 log.cc:824] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:49.633477 8433 log.cc:824] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:49.633857 8434 log.cc:824] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:49.635118 8432 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: No bootstrap required, opened a new log
I20260807 08:29:49.635304 8432 ts_tablet_manager.cc:1397] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:29:49.635557 8434 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: No bootstrap required, opened a new log
I20260807 08:29:49.635720 8434 ts_tablet_manager.cc:1397] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:29:49.638510 8432 raft_consensus.cc:348] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.638593 8434 raft_consensus.cc:348] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.638780 8432 raft_consensus.cc:374] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:49.638808 8434 raft_consensus.cc:374] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:49.638839 8432 raft_consensus.cc:729] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8176d1a69f094511ab13f9c0fbc9e340, State: Initialized, Role: FOLLOWER
I20260807 08:29:49.638868 8434 raft_consensus.cc:729] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47418e13027b45f0b832f964614d6b21, State: Initialized, Role: FOLLOWER
I20260807 08:29:49.639245 8434 consensus_queue.cc:260] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.639245 8432 consensus_queue.cc:260] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.639842 8212 heartbeater.cc:503] Master 127.1.157.126:39611 was elected leader, sending a full tablet report...
I20260807 08:29:49.639842 8032 heartbeater.cc:503] Master 127.1.157.126:39611 was elected leader, sending a full tablet report...
I20260807 08:29:49.640164 8432 ts_tablet_manager.cc:1428] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:29:49.640170 8434 ts_tablet_manager.cc:1428] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:29:49.641764 8433 tablet_bootstrap.cc:492] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: No bootstrap required, opened a new log
I20260807 08:29:49.641937 8433 ts_tablet_manager.cc:1397] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260807 08:29:49.645030 8433 raft_consensus.cc:348] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.645229 8433 raft_consensus.cc:374] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:49.645284 8433 raft_consensus.cc:729] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3377e7c63764016af50965dccb3c403, State: Initialized, Role: FOLLOWER
I20260807 08:29:49.645609 8433 consensus_queue.cc:260] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.646166 8401 heartbeater.cc:503] Master 127.1.157.126:39611 was elected leader, sending a full tablet report...
I20260807 08:29:49.646462 8433 ts_tablet_manager.cc:1428] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260807 08:29:49.669960 8033 tablet.cc:2369] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:49.821614 8406 tablet.cc:2369] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:49.867388 8213 tablet.cc:2369] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:49.978259 8440 raft_consensus.cc:482] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:49.978464 8440 raft_consensus.cc:504] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.979540 8440 leader_election.cc:288] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47418e13027b45f0b832f964614d6b21 (127.1.157.65:40839), b3377e7c63764016af50965dccb3c403 (127.1.157.67:39653)
I20260807 08:29:49.983304 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.979907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57811 (local address 127.1.157.67:39653)
0807 08:29:49.980030 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.980034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.980236 (+ 202us) server_negotiation.cc:407] Connection header received
0807 08:29:49.980373 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.980375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.980415 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.980487 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:49.980839 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.981334 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982406 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.982797 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982897 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.982947 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.983115 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.983184 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:29:49.983319 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.979881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:60525 (local address 127.1.157.65:40839)
0807 08:29:49.980018 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:49.980023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:49.980185 (+ 162us) server_negotiation.cc:407] Connection header received
0807 08:29:49.980324 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:49.980328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:49.980378 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:49.980443 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:49.980775 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.981313 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982414 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.982811 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982916 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.982966 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:49.983146 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:29:49.983216 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:29:49.983331 8442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.979756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:40839 (local address 127.1.157.66:60525)
0807 08:29:49.980057 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.980070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.980195 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.980467 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.980470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.980486 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:49.980663 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.980669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.981419 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.981423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.982289 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.982298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.982922 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.983026 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.983044 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.983112 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.983180 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":30,"thread_start_us":91,"threads_started":1}
I20260807 08:29:49.983338 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:49.979785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39653 (local address 127.1.157.66:57811)
0807 08:29:49.980117 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:49.980130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:49.980273 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:49.980510 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:49.980512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:49.980540 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:49.980723 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.980730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.981438 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.981442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.982286 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:49.982295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:49.982909 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:49.982912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:49.983001 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:49.983025 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:29:49.983081 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:29:49.983147 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":70,"spinlock_wait_cycles":1408,"thread_start_us":89,"threads_started":1}
I20260807 08:29:49.983872 7965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48738f0c9bcb4cd48b3797dd96a5d92e" candidate_uuid: "8176d1a69f094511ab13f9c0fbc9e340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47418e13027b45f0b832f964614d6b21" is_pre_election: true
I20260807 08:29:49.984045 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48738f0c9bcb4cd48b3797dd96a5d92e" candidate_uuid: "8176d1a69f094511ab13f9c0fbc9e340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3377e7c63764016af50965dccb3c403" is_pre_election: true
I20260807 08:29:49.984170 7965 raft_consensus.cc:2454] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8176d1a69f094511ab13f9c0fbc9e340 in term 0.
I20260807 08:29:49.984295 8338 raft_consensus.cc:2454] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8176d1a69f094511ab13f9c0fbc9e340 in term 0.
I20260807 08:29:49.984653 8060 leader_election.cc:302] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 47418e13027b45f0b832f964614d6b21, 8176d1a69f094511ab13f9c0fbc9e340, b3377e7c63764016af50965dccb3c403; no voters:
I20260807 08:29:49.984928 8440 raft_consensus.cc:2781] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:49.985001 8440 raft_consensus.cc:482] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:49.985031 8440 raft_consensus.cc:3037] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:49.986008 8440 raft_consensus.cc:504] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.986306 8440 leader_election.cc:288] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [CANDIDATE]: Term 1 election: Requested vote from peers 47418e13027b45f0b832f964614d6b21 (127.1.157.65:40839), b3377e7c63764016af50965dccb3c403 (127.1.157.67:39653)
I20260807 08:29:49.986615 7965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48738f0c9bcb4cd48b3797dd96a5d92e" candidate_uuid: "8176d1a69f094511ab13f9c0fbc9e340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47418e13027b45f0b832f964614d6b21"
I20260807 08:29:49.986642 8338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48738f0c9bcb4cd48b3797dd96a5d92e" candidate_uuid: "8176d1a69f094511ab13f9c0fbc9e340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3377e7c63764016af50965dccb3c403"
I20260807 08:29:49.986724 7965 raft_consensus.cc:3037] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:49.986753 8338 raft_consensus.cc:3037] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:49.987663 7965 raft_consensus.cc:2454] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8176d1a69f094511ab13f9c0fbc9e340 in term 1.
I20260807 08:29:49.987746 8338 raft_consensus.cc:2454] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8176d1a69f094511ab13f9c0fbc9e340 in term 1.
I20260807 08:29:49.987929 8060 leader_election.cc:302] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47418e13027b45f0b832f964614d6b21, 8176d1a69f094511ab13f9c0fbc9e340; no voters:
I20260807 08:29:49.988075 8440 raft_consensus.cc:2781] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:49.988268 8440 raft_consensus.cc:686] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 1 LEADER]: Becoming Leader. State: Replica: 8176d1a69f094511ab13f9c0fbc9e340, State: Running, Role: LEADER
I20260807 08:29:49.988545 8440 consensus_queue.cc:237] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } }
I20260807 08:29:49.991405 7765 catalog_manager.cc:5495] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8176d1a69f094511ab13f9c0fbc9e340 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "8176d1a69f094511ab13f9c0fbc9e340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8176d1a69f094511ab13f9c0fbc9e340" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 } health_report { overall_health: UNKNOWN } } }
I20260807 08:29:50.065910 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.062572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.1.157.66:34625)
0807 08:29:50.062716 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.062720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.062734 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:29:50.062837 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.062839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.062892 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.062979 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:50.063250 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:50.063744 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:50.064579 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:50.064796 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:50.064881 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:50.065164 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:50.065541 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:50.065642 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.065725 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.065781 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:29:50.071219 7965 raft_consensus.cc:1261] T 48738f0c9bcb4cd48b3797dd96a5d92e P 47418e13027b45f0b832f964614d6b21 [term 1 FOLLOWER]: Refusing update from remote peer 8176d1a69f094511ab13f9c0fbc9e340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:50.071218 8338 raft_consensus.cc:1261] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 1 FOLLOWER]: Refusing update from remote peer 8176d1a69f094511ab13f9c0fbc9e340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:50.071863 8444 consensus_queue.cc:1035] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47418e13027b45f0b832f964614d6b21" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 40839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:50.072033 8440 consensus_queue.cc:1035] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3377e7c63764016af50965dccb3c403" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:50.079411 8449 mvcc.cc:205] Tried to move back new op lower bound from 7315830333725360128 to 7315830333395472384. Current Snapshot: MvccSnapshot[applied={T|T < 7315830333725360128}]
I20260807 08:29:50.079015 8448 mvcc.cc:205] Tried to move back new op lower bound from 7315830333725360128 to 7315830333395472384. Current Snapshot: MvccSnapshot[applied={T|T < 7315830333725360128}]
I20260807 08:29:50.087016 7766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:50.087178 7766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:50.089432 7766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c17dd0b1f23d495b9ab99e0099f194a8: Sending DeleteTablet for 3 replicas of tablet 48738f0c9bcb4cd48b3797dd96a5d92e
I20260807 08:29:50.090164 8121 tablet_service.cc:1514] Processing DeleteTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:50 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37610
I20260807 08:29:50.090255 7938 tablet_service.cc:1514] Processing DeleteTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:50 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:55074
I20260807 08:29:50.090250 8308 tablet_service.cc:1514] Processing DeleteTablet for tablet 48738f0c9bcb4cd48b3797dd96a5d92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:50 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60042
I20260807 08:29:50.090685 8467 tablet_replica.cc:330] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: stopping tablet replica
I20260807 08:29:50.090936 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 7851
I20260807 08:29:50.090963 8467 raft_consensus.cc:2229] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:50.091255 8467 raft_consensus.cc:2258] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:50.098811 8469 tablet_replica.cc:330] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: stopping tablet replica
I20260807 08:29:50.099119 8469 raft_consensus.cc:2229] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:50.099380 8467 ts_tablet_manager.cc:1905] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:50.099514 8469 raft_consensus.cc:2258] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:50.101573 8469 ts_tablet_manager.cc:1905] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:50.102331 8467 ts_tablet_manager.cc:1918] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:50.102514 8467 log.cc:1196] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/48738f0c9bcb4cd48b3797dd96a5d92e
I20260807 08:29:50.102896 8467 ts_tablet_manager.cc:1939] T 48738f0c9bcb4cd48b3797dd96a5d92e P b3377e7c63764016af50965dccb3c403: Deleting consensus metadata
I20260807 08:29:50.104115 7749 catalog_manager.cc:4841] TS b3377e7c63764016af50965dccb3c403 (127.1.157.67:39653): tablet 48738f0c9bcb4cd48b3797dd96a5d92e (table test-table [id=0c22e19b7e1840c79e2487586fb33c67]) successfully deleted
I20260807 08:29:50.104148 8469 ts_tablet_manager.cc:1918] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:50.104226 8469 log.cc:1196] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/48738f0c9bcb4cd48b3797dd96a5d92e
I20260807 08:29:50.104491 8469 ts_tablet_manager.cc:1939] T 48738f0c9bcb4cd48b3797dd96a5d92e P 8176d1a69f094511ab13f9c0fbc9e340: Deleting consensus metadata
I20260807 08:29:50.105484 7750 catalog_manager.cc:4841] TS 8176d1a69f094511ab13f9c0fbc9e340 (127.1.157.66:34625): tablet 48738f0c9bcb4cd48b3797dd96a5d92e (table test-table [id=0c22e19b7e1840c79e2487586fb33c67]) successfully deleted
W20260807 08:29:50.106766 7750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:40839 (error 108)
I20260807 08:29:50.106992 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8037
I20260807 08:29:50.114739 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.114489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:40839 (local address 127.0.0.1:55086)
0807 08:29:50.114603 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.114667 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:40839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:29:50.114892 7750 catalog_manager.cc:4584] TS 47418e13027b45f0b832f964614d6b21 (127.1.157.65:40839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48738f0c9bcb4cd48b3797dd96a5d92e: Network error: Client connection negotiation failed: client connection to 127.1.157.65:40839: connect: Connection refused (error 111)
I20260807 08:29:50.119522 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8218
I20260807 08:29:50.139144 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 7721
2026-08-07T08:29:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2617 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T08:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:50Z Disabled control of system clock
I20260807 08:29:50.171043 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44455
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:33427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:29:50.275530 8475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:50.275746 8475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:50.275779 8475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:50.279603 8475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:50.279650 8475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:50.279664 8475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:50.279678 8475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:50.279696 8475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:50.284230 8475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:44455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:50.285111 8475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:50.286403 8475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:29:50.294135 8475 server_base.cc:1034] running on GCE node
W20260807 08:29:50.294101 8482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.294121 8481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.294288 8484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:50.294850 8475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:50.295892 8475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:50.297040 8475 hybrid_clock.cc:648] HybridClock initialized: now 1786091390297019 us; error 33 us; skew 500 ppm
I20260807 08:29:50.298786 8475 webserver.cc:460] Webserver started at http://127.1.157.126:34693/ using document root <none> and password file <none>
I20260807 08:29:50.299294 8475 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:50.299353 8475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:50.299542 8475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:50.301493 8475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "38d3a9c1399b463e8874d4921339c64e"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "204a01dd349709bcbf9f7bac2ac2d1b2"
server_key_iv: "81dead8b1b786390c42c54673034e939"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.302039 8475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "38d3a9c1399b463e8874d4921339c64e"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "204a01dd349709bcbf9f7bac2ac2d1b2"
server_key_iv: "81dead8b1b786390c42c54673034e939"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.305531 8475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:29:50.307781 8493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.308894 8475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:50.308998 8475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "38d3a9c1399b463e8874d4921339c64e"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "204a01dd349709bcbf9f7bac2ac2d1b2"
server_key_iv: "81dead8b1b786390c42c54673034e939"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.309100 8475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:50.313453 8475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:50.314013 8475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:50.314162 8475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:50.320698 8475 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:44455
I20260807 08:29:50.320708 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:44455 every 8 connection(s)
I20260807 08:29:50.321681 8475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:50.324745 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:50.325986 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8475
I20260807 08:29:50.326102 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:50.326427 1653 external_mini_cluster.cc:1428] Setting key 0a602bf71ebd239695b5518600e8fb98
I20260807 08:29:50.330680 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: Bootstrap starting.
I20260807 08:29:50.331597 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.327767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.1.157.126:44455)
0807 08:29:50.328369 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.328383 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.328416 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:29:50.329048 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.329064 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.329382 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.329667 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.330419 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.330425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.330440 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.330450 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.330503 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.330519 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.330639 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.330731 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.330968 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
I20260807 08:29:50.332896 8567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:50.333644 8567 log.cc:824] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:50.335364 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: No bootstrap required, opened a new log
I20260807 08:29:50.338616 8567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } }
I20260807 08:29:50.338843 8567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:50.338904 8567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d3a9c1399b463e8874d4921339c64e, State: Initialized, Role: FOLLOWER
I20260807 08:29:50.339293 8567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } }
I20260807 08:29:50.339421 8567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:50.339464 8567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:50.339532 8567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:50.340270 8567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } }
I20260807 08:29:50.340541 8567 leader_election.cc:302] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d3a9c1399b463e8874d4921339c64e; no voters:
I20260807 08:29:50.340786 8567 leader_election.cc:288] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:50.340868 8573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:50.341090 8573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [term 1 LEADER]: Becoming Leader. State: Replica: 38d3a9c1399b463e8874d4921339c64e, State: Running, Role: LEADER
I20260807 08:29:50.341387 8573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } }
I20260807 08:29:50.341876 8567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:50.342905 8576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d3a9c1399b463e8874d4921339c64e. Latest consensus state: current_term: 1 leader_uuid: "38d3a9c1399b463e8874d4921339c64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } } }
I20260807 08:29:50.342892 8575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d3a9c1399b463e8874d4921339c64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d3a9c1399b463e8874d4921339c64e" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44455 } } }
I20260807 08:29:50.343036 8575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:50.343036 8576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:50.343416 8586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:50.346129 8586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:50.350859 8586 catalog_manager.cc:1347] Generated new cluster ID: d74b7d4318d049149872e20806ed89a5
I20260807 08:29:50.350924 8586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:50.366896 8586 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:50.367208 8586 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:50.373214 8586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: Generated new TSK 0
I20260807 08:29:50.373760 8586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:29:50.377439 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--builtin_ntp_servers=127.1.157.84:33427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:29:50.483999 8600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:50.484223 8600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:50.484278 8600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:50.488205 8600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:50.488250 8600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:50.488324 8600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:50.492822 8600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:50.493739 8600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:50.495021 8600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:50.503855 8607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.503855 8608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.503849 8612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:50.504226 8600 server_base.cc:1034] running on GCE node
I20260807 08:29:50.504578 8600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:50.505146 8600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:50.506294 8600 hybrid_clock.cc:648] HybridClock initialized: now 1786091390506279 us; error 26 us; skew 500 ppm
I20260807 08:29:50.508018 8600 webserver.cc:460] Webserver started at http://127.1.157.65:39431/ using document root <none> and password file <none>
I20260807 08:29:50.508520 8600 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:50.508580 8600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:50.508782 8600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:50.510751 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "aa73a7e2b7a547828f8d283b90c0e85f"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "6ef73ebc28c989c1743814b327e3236b"
server_key_iv: "fad4d856a4efe4ae0c26eab4f076c993"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.511284 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "aa73a7e2b7a547828f8d283b90c0e85f"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "6ef73ebc28c989c1743814b327e3236b"
server_key_iv: "fad4d856a4efe4ae0c26eab4f076c993"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.514868 8600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:29:50.517098 8623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.518301 8600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:50.518432 8600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "aa73a7e2b7a547828f8d283b90c0e85f"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "6ef73ebc28c989c1743814b327e3236b"
server_key_iv: "fad4d856a4efe4ae0c26eab4f076c993"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.518530 8600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:50.522841 8600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:50.523404 8600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:50.523545 8600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:50.524040 8600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:50.524899 8600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:50.524943 8600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.524981 8600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:50.525009 8600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.532863 8600 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:33867
I20260807 08:29:50.532886 8784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:33867 every 8 connection(s)
I20260807 08:29:50.533919 8600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:50.539400 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.535608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:35083 (local address 127.1.157.126:44455)
0807 08:29:50.535737 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.535740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.536722 (+ 982us) server_negotiation.cc:407] Connection header received
0807 08:29:50.537991 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.537993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.538030 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.538103 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.539052 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.539055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.539058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.539068 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.539087 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.539095 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.539187 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.539265 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.539298 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:29:50.540145 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.535970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44455 (local address 127.1.157.65:35083)
0807 08:29:50.536580 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.536618 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.537813 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.538238 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.538244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.538325 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.538827 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.538829 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.538865 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.538928 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.539174 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.539178 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.539181 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.539295 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.539468 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":123,"threads_started":1}
I20260807 08:29:50.541209 8785 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44455
I20260807 08:29:50.541402 8785 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:50.541759 8785 heartbeater.cc:511] Master 127.1.157.126:44455 requested a full tablet report, sending...
I20260807 08:29:50.543133 8517 ts_manager.cc:194] Registered new tserver with Master: aa73a7e2b7a547828f8d283b90c0e85f (127.1.157.65:33867)
I20260807 08:29:50.543826 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8600
I20260807 08:29:50.543948 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:50.544258 1653 external_mini_cluster.cc:1428] Setting key 44dd149602e3a3eb5e123e990dc90941
I20260807 08:29:50.545953 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--builtin_ntp_servers=127.1.157.84:33427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 08:29:50.653041 8788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:50.653250 8788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:50.653286 8788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:50.657097 8788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:50.657143 8788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:50.657193 8788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:50.661631 8788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:50.662480 8788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:50.663630 8788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:50.672103 8797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.672111 8793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:50.672215 8788 server_base.cc:1034] running on GCE node
W20260807 08:29:50.672112 8794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:50.672777 8788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:50.673280 8788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:50.674403 8788 hybrid_clock.cc:648] HybridClock initialized: now 1786091390674390 us; error 24 us; skew 500 ppm
I20260807 08:29:50.676026 8788 webserver.cc:460] Webserver started at http://127.1.157.66:34255/ using document root <none> and password file <none>
I20260807 08:29:50.676507 8788 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:50.676566 8788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:50.676745 8788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:50.678607 8788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "dd3d6feb897a4f5487c70d380f03ba5a"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "882f7c7ddb384133de0856dd57a7e056"
server_key_iv: "5826ea783a08385a05e58f6a851bf37a"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.679105 8788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "dd3d6feb897a4f5487c70d380f03ba5a"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "882f7c7ddb384133de0856dd57a7e056"
server_key_iv: "5826ea783a08385a05e58f6a851bf37a"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.682389 8788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:29:50.684530 8806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.685573 8788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:29:50.685667 8788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "dd3d6feb897a4f5487c70d380f03ba5a"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "882f7c7ddb384133de0856dd57a7e056"
server_key_iv: "5826ea783a08385a05e58f6a851bf37a"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.685762 8788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:50.689800 8788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:50.690433 8788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:50.690586 8788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:50.691183 8788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:50.691985 8788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:50.692032 8788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.692080 8788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:50.692108 8788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.699918 8788 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:44023
I20260807 08:29:50.699945 8967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:44023 every 8 connection(s)
I20260807 08:29:50.700790 8788 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:50.701831 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8788
I20260807 08:29:50.701946 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:50.702198 1653 external_mini_cluster.cc:1428] Setting key a2055657f1126b19f4227cf77d8dca7c
I20260807 08:29:50.704172 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--builtin_ntp_servers=127.1.157.84:33427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 08:29:50.706318 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.703700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:50137 (local address 127.1.157.126:44455)
0807 08:29:50.703820 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.703824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.703840 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:29:50.704813 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.704817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.704858 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.704937 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.705955 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.705958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.705960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.705970 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.705990 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.705997 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.706066 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.706189 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.706221 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:29:50.707350 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.702801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44455 (local address 127.1.157.66:50137)
0807 08:29:50.703449 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.703495 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.704621 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.705069 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.705075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.705152 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.705657 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.705659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.705711 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.705810 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.706069 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.706075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.706080 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.706236 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.706500 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":96,"threads_started":1}
I20260807 08:29:50.708365 8968 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44455
I20260807 08:29:50.708560 8968 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:50.708916 8968 heartbeater.cc:511] Master 127.1.157.126:44455 requested a full tablet report, sending...
I20260807 08:29:50.709858 8517 ts_manager.cc:194] Registered new tserver with Master: dd3d6feb897a4f5487c70d380f03ba5a (127.1.157.66:44023)
W20260807 08:29:50.808961 8972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:50.809171 8972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:50.809207 8972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:50.813071 8972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:50.813117 8972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:50.813167 8972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:50.817665 8972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:50.818502 8972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:50.819674 8972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:50.828213 8981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.828214 8982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:50.828272 8984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:50.828512 8972 server_base.cc:1034] running on GCE node
I20260807 08:29:50.828853 8972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:50.829396 8972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:50.830543 8972 hybrid_clock.cc:648] HybridClock initialized: now 1786091390830529 us; error 28 us; skew 500 ppm
I20260807 08:29:50.832185 8972 webserver.cc:460] Webserver started at http://127.1.157.67:32947/ using document root <none> and password file <none>
I20260807 08:29:50.832674 8972 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:50.832737 8972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:50.832904 8972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:50.834808 8972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "73364562359c4e51a3a53e6a67ddbe30"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "de53c2d71577986281ed6e7e53d76c12"
server_key_iv: "fde785c0727a93f38d45d0863fdb156c"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.835319 8972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "73364562359c4e51a3a53e6a67ddbe30"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "de53c2d71577986281ed6e7e53d76c12"
server_key_iv: "fde785c0727a93f38d45d0863fdb156c"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.838537 8972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:50.840705 8991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.841768 8972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:50.841872 8972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "73364562359c4e51a3a53e6a67ddbe30"
format_stamp: "Formatted at 2026-08-07 08:29:50 on dist-test-slave-6zbq"
server_key: "de53c2d71577986281ed6e7e53d76c12"
server_key_iv: "fde785c0727a93f38d45d0863fdb156c"
server_key_version: "encryptionkey@0"
I20260807 08:29:50.841969 8972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:50.846439 8972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:50.846956 8972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:50.847105 8972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:50.847595 8972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:50.848397 8972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:50.848441 8972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.848487 8972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:50.848516 8972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:50.856309 8972 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:45073
I20260807 08:29:50.856365 9151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:45073 every 8 connection(s)
I20260807 08:29:50.857131 8972 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:50.860466 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 8972
I20260807 08:29:50.860566 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:50.860810 1653 external_mini_cluster.cc:1428] Setting key f479e8fd3f5db248abc7445479fd4638
I20260807 08:29:50.862514 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.858631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:33303 (local address 127.1.157.126:44455)
0807 08:29:50.858778 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.858783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.859717 (+ 934us) server_negotiation.cc:407] Connection header received
0807 08:29:50.860914 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.860918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.860962 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.861036 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.862071 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.862074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.862077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.862091 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.862115 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.862125 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.862202 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.862325 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.862387 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:29:50.863312 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.858993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44455 (local address 127.1.157.67:33303)
0807 08:29:50.859591 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.859628 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.860725 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.861194 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.861200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.861288 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.861832 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.861833 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.861870 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.861935 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.862198 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.862206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.862211 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.862366 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.862554 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260807 08:29:50.864243 9152 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44455
I20260807 08:29:50.864445 9152 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:50.864802 9152 heartbeater.cc:511] Master 127.1.157.126:44455 requested a full tablet report, sending...
I20260807 08:29:50.865646 8517 ts_manager.cc:194] Registered new tserver with Master: 73364562359c4e51a3a53e6a67ddbe30 (127.1.157.67:45073)
I20260807 08:29:50.874275 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:29:50.877259 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.876047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49828 (local address 127.1.157.126:44455)
0807 08:29:50.876153 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.876156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.876324 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:29:50.876414 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.876416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.876445 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.876503 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.877000 (+ 497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.877003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.877005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.877015 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.877032 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.877040 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.877101 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.877137 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.877166 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:50.879609 8517 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:50.882731 8517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:50.893162 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33867 (local address 127.0.0.1:57276)
0807 08:29:50.891385 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.891398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.891484 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.891893 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.891896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.891909 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.892571 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.892572 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.892582 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.892613 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.892912 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.892916 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.892919 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.892999 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.893034 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":77,"threads_started":1}
I20260807 08:29:50.893237 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57276 (local address 127.1.157.65:33867)
0807 08:29:50.891253 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.891264 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.891492 (+ 228us) server_negotiation.cc:407] Connection header received
0807 08:29:50.891575 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.891580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.891754 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.891884 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.892726 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.892732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.892742 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.892754 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.892795 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.892824 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.892921 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.893009 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.893075 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":136,"threads_started":1}
I20260807 08:29:50.893616 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.1.157.67:45073)
0807 08:29:50.890656 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.890668 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.890692 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:29:50.890828 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.890834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.891024 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.891154 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.892557 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.892562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.892571 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.892581 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.892618 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.892639 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.892744 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.893394 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.893463 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":158,"threads_started":1}
I20260807 08:29:50.893661 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:45073 (local address 127.0.0.1:33486)
0807 08:29:50.890529 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.890576 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.890683 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.891768 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.891774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.891848 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.892341 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.892343 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.892366 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.892433 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.892791 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.892799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.892805 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.892911 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.892976 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":67,"threads_started":1}
I20260807 08:29:50.894294 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44023 (local address 127.0.0.1:53468)
0807 08:29:50.891927 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:50.891940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:50.892015 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:50.892947 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:50.892949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:50.892963 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:50.893522 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:50.893524 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:50.893532 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:50.893555 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:50.894066 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:50.894069 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:50.894072 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:29:50.894148 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:29:50.894184 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":87,"threads_started":1}
I20260807 08:29:50.894438 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:50.890932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.1.157.66:44023)
0807 08:29:50.891797 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:50.891806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:50.892110 (+ 304us) server_negotiation.cc:407] Connection header received
0807 08:29:50.892160 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:50.892163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:50.892314 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:50.892434 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:50.893673 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:50.893679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:50.893690 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:50.893703 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:50.893886 (+ 183us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:50.893922 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:50.894023 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:50.894191 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:29:50.894256 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":126,"threads_started":1}
I20260807 08:29:50.896289 9064 tablet_service.cc:1467] Processing CreateTablet for tablet dc7a5f6a1b264e73998da5134c49f095 (DEFAULT_TABLE table=test-table [id=05d7392eb8c340de9a854db4680494f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:50.896206 8686 tablet_service.cc:1467] Processing CreateTablet for tablet dc7a5f6a1b264e73998da5134c49f095 (DEFAULT_TABLE table=test-table [id=05d7392eb8c340de9a854db4680494f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:50.897171 9064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7a5f6a1b264e73998da5134c49f095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:50.896826 8872 tablet_service.cc:1467] Processing CreateTablet for tablet dc7a5f6a1b264e73998da5134c49f095 (DEFAULT_TABLE table=test-table [id=05d7392eb8c340de9a854db4680494f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:50.897221 8686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7a5f6a1b264e73998da5134c49f095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:50.897713 8872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7a5f6a1b264e73998da5134c49f095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:50.903303 9182 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Bootstrap starting.
I20260807 08:29:50.904815 9186 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Bootstrap starting.
I20260807 08:29:50.905517 9182 tablet_bootstrap.cc:654] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:50.906279 9182 log.cc:824] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:50.906571 9186 tablet_bootstrap.cc:654] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:50.906978 9183 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Bootstrap starting.
I20260807 08:29:50.907311 9186 log.cc:824] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:50.908609 9183 tablet_bootstrap.cc:654] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:50.908977 9186 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: No bootstrap required, opened a new log
I20260807 08:29:50.909159 9186 ts_tablet_manager.cc:1397] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:29:50.909251 9183 log.cc:824] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:50.911087 9182 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: No bootstrap required, opened a new log
I20260807 08:29:50.911087 9183 tablet_bootstrap.cc:492] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: No bootstrap required, opened a new log
I20260807 08:29:50.911271 9183 ts_tablet_manager.cc:1397] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 08:29:50.911271 9182 ts_tablet_manager.cc:1397] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 08:29:50.912989 9186 raft_consensus.cc:348] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.913264 9186 raft_consensus.cc:374] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:50.913318 9186 raft_consensus.cc:729] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa73a7e2b7a547828f8d283b90c0e85f, State: Initialized, Role: FOLLOWER
I20260807 08:29:50.913749 9186 consensus_queue.cc:260] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.914330 8785 heartbeater.cc:503] Master 127.1.157.126:44455 was elected leader, sending a full tablet report...
I20260807 08:29:50.914712 9186 ts_tablet_manager.cc:1428] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:29:50.915494 9183 raft_consensus.cc:348] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.915494 9182 raft_consensus.cc:348] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.915743 9183 raft_consensus.cc:374] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:50.915745 9182 raft_consensus.cc:374] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:50.915799 9182 raft_consensus.cc:729] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73364562359c4e51a3a53e6a67ddbe30, State: Initialized, Role: FOLLOWER
I20260807 08:29:50.915799 9183 raft_consensus.cc:729] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3d6feb897a4f5487c70d380f03ba5a, State: Initialized, Role: FOLLOWER
I20260807 08:29:50.916221 9182 consensus_queue.cc:260] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.916221 9183 consensus_queue.cc:260] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:50.916847 9152 heartbeater.cc:503] Master 127.1.157.126:44455 was elected leader, sending a full tablet report...
I20260807 08:29:50.917130 9182 ts_tablet_manager.cc:1428] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:29:50.917121 8968 heartbeater.cc:503] Master 127.1.157.126:44455 was elected leader, sending a full tablet report...
I20260807 08:29:50.918051 9183 ts_tablet_manager.cc:1428] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 08:29:50.951632 8970 tablet.cc:2369] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:51.035055 8786 tablet.cc:2369] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:29:51.108088 9154 tablet.cc:2369] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:51.110273 9190 raft_consensus.cc:482] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:51.110478 9190 raft_consensus.cc:504] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:51.111529 9190 leader_election.cc:288] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73364562359c4e51a3a53e6a67ddbe30 (127.1.157.67:45073), dd3d6feb897a4f5487c70d380f03ba5a (127.1.157.66:44023)
I20260807 08:29:51.113476 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.111851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52257 (local address 127.1.157.67:45073)
0807 08:29:51.111994 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.111999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.112089 (+ 90us) server_negotiation.cc:407] Connection header received
0807 08:29:51.112247 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.112250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.112289 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.112342 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:51.113061 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:51.113065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:51.113068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:51.113083 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:51.113106 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:51.113117 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:51.113238 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:51.113323 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:29:51.113365 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:29:51.113533 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.111734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:45073 (local address 127.1.157.65:52257)
0807 08:29:51.112006 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:51.112019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:51.112126 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:51.112456 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:51.112458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:51.112472 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:51.112909 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:51.112911 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:51.112923 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:51.112950 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:51.113220 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:51.113224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:51.113227 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:29:51.113325 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 08:29:51.113364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260807 08:29:51.113986 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.112129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44023 (local address 127.1.157.65:48347)
0807 08:29:51.112350 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:51.112363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:51.112445 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:51.112916 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:51.112919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:51.112932 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:51.113352 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:29:51.113354 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:51.113363 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:29:51.113390 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:51.113733 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:51.113736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:51.113739 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:29:51.113815 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:29:51.113851 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260807 08:29:51.113996 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.112259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:48347 (local address 127.1.157.66:44023)
0807 08:29:51.112414 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.112419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.112433 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:29:51.112524 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.112527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.112561 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.112653 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:51.113514 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:51.113518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:51.113521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:51.113533 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:51.113557 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:51.113568 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:51.113686 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:51.113848 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 08:29:51.113892 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 08:29:51.114111 9093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7a5f6a1b264e73998da5134c49f095" candidate_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73364562359c4e51a3a53e6a67ddbe30" is_pre_election: true
I20260807 08:29:51.114460 9093 raft_consensus.cc:2454] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa73a7e2b7a547828f8d283b90c0e85f in term 0.
I20260807 08:29:51.114624 8899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7a5f6a1b264e73998da5134c49f095" candidate_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" is_pre_election: true
I20260807 08:29:51.114918 8899 raft_consensus.cc:2454] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa73a7e2b7a547828f8d283b90c0e85f in term 0.
I20260807 08:29:51.114915 8628 leader_election.cc:302] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73364562359c4e51a3a53e6a67ddbe30, aa73a7e2b7a547828f8d283b90c0e85f; no voters:
I20260807 08:29:51.115155 9190 raft_consensus.cc:2781] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:51.115252 9190 raft_consensus.cc:482] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:51.115305 9190 raft_consensus.cc:3037] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:51.116304 9190 raft_consensus.cc:504] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:51.116605 9190 leader_election.cc:288] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [CANDIDATE]: Term 1 election: Requested vote from peers 73364562359c4e51a3a53e6a67ddbe30 (127.1.157.67:45073), dd3d6feb897a4f5487c70d380f03ba5a (127.1.157.66:44023)
I20260807 08:29:51.116967 8899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7a5f6a1b264e73998da5134c49f095" candidate_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3d6feb897a4f5487c70d380f03ba5a"
I20260807 08:29:51.116971 9093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7a5f6a1b264e73998da5134c49f095" candidate_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73364562359c4e51a3a53e6a67ddbe30"
I20260807 08:29:51.117064 8899 raft_consensus.cc:3037] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:51.117074 9093 raft_consensus.cc:3037] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:51.118196 9093 raft_consensus.cc:2454] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa73a7e2b7a547828f8d283b90c0e85f in term 1.
I20260807 08:29:51.118207 8899 raft_consensus.cc:2454] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa73a7e2b7a547828f8d283b90c0e85f in term 1.
I20260807 08:29:51.118500 8628 leader_election.cc:302] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73364562359c4e51a3a53e6a67ddbe30, aa73a7e2b7a547828f8d283b90c0e85f; no voters:
I20260807 08:29:51.118662 9190 raft_consensus.cc:2781] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:51.118863 9190 raft_consensus.cc:686] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 1 LEADER]: Becoming Leader. State: Replica: aa73a7e2b7a547828f8d283b90c0e85f, State: Running, Role: LEADER
I20260807 08:29:51.119187 9190 consensus_queue.cc:237] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } }
I20260807 08:29:51.122407 8516 catalog_manager.cc:5495] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f reported cstate change: term changed from 0 to 1, leader changed from <none> to aa73a7e2b7a547828f8d283b90c0e85f (127.1.157.65). New cstate: current_term: 1 leader_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa73a7e2b7a547828f8d283b90c0e85f" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33867 } health_report { overall_health: HEALTHY } } }
I20260807 08:29:51.181119 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.179856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57278 (local address 127.1.157.65:33867)
0807 08:29:51.179986 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.179989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.180003 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:29:51.180098 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.180100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.180163 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.180229 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:51.180775 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:51.180778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:51.180780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:51.180790 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:51.180809 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:51.180817 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:51.180883 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:51.180939 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 08:29:51.180972 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:29:51.186570 9093 raft_consensus.cc:1261] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 1 FOLLOWER]: Refusing update from remote peer aa73a7e2b7a547828f8d283b90c0e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:51.186571 8899 raft_consensus.cc:1261] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 1 FOLLOWER]: Refusing update from remote peer aa73a7e2b7a547828f8d283b90c0e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:51.187208 9196 consensus_queue.cc:1035] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd3d6feb897a4f5487c70d380f03ba5a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:51.187374 9190 consensus_queue.cc:1035] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73364562359c4e51a3a53e6a67ddbe30" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 45073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:51.195660 9199 mvcc.cc:205] Tried to move back new op lower bound from 7315830338293563392 to 7315830338026946560. Current Snapshot: MvccSnapshot[applied={T|T < 7315830338293563392 or (T in {7315830338293563392})}]
I20260807 08:29:51.196372 9200 mvcc.cc:205] Tried to move back new op lower bound from 7315830338293563392 to 7315830338026946560. Current Snapshot: MvccSnapshot[applied={T|T < 7315830338293563392 or (T in {7315830338293563392})}]
I20260807 08:29:51.201807 8515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:51.201957 8515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:51.204294 8515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38d3a9c1399b463e8874d4921339c64e: Sending DeleteTablet for 3 replicas of tablet dc7a5f6a1b264e73998da5134c49f095
I20260807 08:29:51.204973 8686 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7a5f6a1b264e73998da5134c49f095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:51 UTC) from {username='slave'} at 127.0.0.1:57276
I20260807 08:29:51.205158 8872 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7a5f6a1b264e73998da5134c49f095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:51 UTC) from {username='slave'} at 127.0.0.1:53468
I20260807 08:29:51.205173 9064 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7a5f6a1b264e73998da5134c49f095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:51 UTC) from {username='slave'} at 127.0.0.1:33486
I20260807 08:29:51.205410 9215 tablet_replica.cc:330] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f: stopping tablet replica
I20260807 08:29:51.205575 9216 tablet_replica.cc:330] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: stopping tablet replica
I20260807 08:29:51.205683 9215 raft_consensus.cc:2229] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:51.205862 9216 raft_consensus.cc:2229] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:51.206014 9215 raft_consensus.cc:2258] T dc7a5f6a1b264e73998da5134c49f095 P aa73a7e2b7a547828f8d283b90c0e85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:51.206054 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8600
I20260807 08:29:51.206223 9216 raft_consensus.cc:2258] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:51.207649 9216 ts_tablet_manager.cc:1905] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:51.209656 9216 ts_tablet_manager.cc:1918] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:51.209740 9216 log.cc:1196] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/dc7a5f6a1b264e73998da5134c49f095
I20260807 08:29:51.210145 9216 ts_tablet_manager.cc:1939] T dc7a5f6a1b264e73998da5134c49f095 P dd3d6feb897a4f5487c70d380f03ba5a: Deleting consensus metadata
I20260807 08:29:51.211408 8497 catalog_manager.cc:4841] TS dd3d6feb897a4f5487c70d380f03ba5a (127.1.157.66:44023): tablet dc7a5f6a1b264e73998da5134c49f095 (table test-table [id=05d7392eb8c340de9a854db4680494f5]) successfully deleted
I20260807 08:29:51.214511 9217 tablet_replica.cc:330] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: stopping tablet replica
I20260807 08:29:51.214748 9217 raft_consensus.cc:2229] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:51.214988 9217 raft_consensus.cc:2258] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:51.216032 9217 ts_tablet_manager.cc:1905] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:51.218014 9217 ts_tablet_manager.cc:1918] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:51.218098 9217 log.cc:1196] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/dc7a5f6a1b264e73998da5134c49f095
I20260807 08:29:51.218433 9217 ts_tablet_manager.cc:1939] T dc7a5f6a1b264e73998da5134c49f095 P 73364562359c4e51a3a53e6a67ddbe30: Deleting consensus metadata
W20260807 08:29:51.218485 8496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:33867 (error 108)
I20260807 08:29:51.218591 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8788
I20260807 08:29:51.227659 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.226790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33867 (local address 127.0.0.1:57292)
0807 08:29:51.227506 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:51.227580 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":629}
W20260807 08:29:51.227833 8496 catalog_manager.cc:4584] TS aa73a7e2b7a547828f8d283b90c0e85f (127.1.157.65:33867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc7a5f6a1b264e73998da5134c49f095: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33867: connect: Connection refused (error 111)
I20260807 08:29:51.230571 8497 catalog_manager.cc:4841] TS 73364562359c4e51a3a53e6a67ddbe30 (127.1.157.67:45073): tablet dc7a5f6a1b264e73998da5134c49f095 (table test-table [id=05d7392eb8c340de9a854db4680494f5]) successfully deleted
I20260807 08:29:51.233881 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8972
I20260807 08:29:51.243763 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 8475
2026-08-07T08:29:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1107 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 08:29:51.271138 1653 mini_oidc.cc:152] Starting JWKS server
I20260807 08:29:51.272904 1653 webserver.cc:460] Webserver started at https://127.0.0.1:41013/ using document root <none> and password file <none>
I20260807 08:29:51.273066 1653 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:29:51.273321 1653 webserver.cc:460] Webserver started at http://127.0.0.1:45265/ using document root <none> and password file <none>
2026-08-07T08:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:51Z Disabled control of system clock
I20260807 08:29:51.282049 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:41275
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:44573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:41275
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41013/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/testchainca.cert with env {}
W20260807 08:29:51.387465 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:51.387681 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:51.387723 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:51.391566 9225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:51.391611 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:51.391626 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:51.391645 9225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:51.391673 9225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:29:51.391700 9225 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:29:51.391722 9225 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41013/jwks.json
I20260807 08:29:51.396733 9225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:41275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:41275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41013/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:51.397574 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:51.398473 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:51.405889 9232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:51.405926 9225 server_base.cc:1034] running on GCE node
W20260807 08:29:51.405866 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:51.405879 9236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:51.406531 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:51.407485 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:51.408638 9225 hybrid_clock.cc:648] HybridClock initialized: now 1786091391408613 us; error 33 us; skew 500 ppm
I20260807 08:29:51.410303 9225 webserver.cc:460] Webserver started at http://127.1.157.126:39313/ using document root <none> and password file <none>
I20260807 08:29:51.410820 9225 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:51.410877 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:51.411059 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:51.412842 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "ba88bf3ec5a64ed28530074969355316"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "790f5fcdc2432d6ae917cc79936c8738"
server_key_iv: "dacff7aa00bd04c05dc65c2640f976e1"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.413301 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "ba88bf3ec5a64ed28530074969355316"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "790f5fcdc2432d6ae917cc79936c8738"
server_key_iv: "dacff7aa00bd04c05dc65c2640f976e1"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.416510 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 08:29:51.418556 9242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:51.419592 9225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:51.419705 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "ba88bf3ec5a64ed28530074969355316"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "790f5fcdc2432d6ae917cc79936c8738"
server_key_iv: "dacff7aa00bd04c05dc65c2640f976e1"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.419790 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:51.432914 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:51.433449 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:51.433598 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:51.439632 9225 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:41275
I20260807 08:29:51.439654 9322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:41275 every 8 connection(s)
I20260807 08:29:51.440435 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:51.443228 9323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:51.446987 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9225
I20260807 08:29:51.447089 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:51.447314 1653 external_mini_cluster.cc:1428] Setting key 532575e7e8690740c33de653b946ad12
I20260807 08:29:51.448273 9323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316: Bootstrap starting.
I20260807 08:29:51.450722 9323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:51.451551 9323 log.cc:824] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:51.453317 9323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316: No bootstrap required, opened a new log
I20260807 08:29:51.456725 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.448481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.1.157.126:41275)
0807 08:29:51.449081 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.449098 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.449135 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:29:51.449906 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.449928 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.450396 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.450741 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:51.451166 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.452410 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.453280 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.453948 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.454110 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.454794 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:51.454800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:51.454816 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:51.455271 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:51.455337 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:51.455362 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:51.455460 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:51.455542 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 08:29:51.455806 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
I20260807 08:29:51.457497 9323 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } }
I20260807 08:29:51.457747 9323 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:51.457813 9323 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba88bf3ec5a64ed28530074969355316, State: Initialized, Role: FOLLOWER
I20260807 08:29:51.458206 9323 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } }
I20260807 08:29:51.458320 9323 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:51.458398 9323 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:51.458496 9323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:51.459379 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:29:51.459405 9323 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } }
I20260807 08:29:51.459707 9323 leader_election.cc:302] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba88bf3ec5a64ed28530074969355316; no voters:
I20260807 08:29:51.459957 9323 leader_election.cc:288] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:51.460070 9329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:51.460289 9329 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [term 1 LEADER]: Becoming Leader. State: Replica: ba88bf3ec5a64ed28530074969355316, State: Running, Role: LEADER
I20260807 08:29:51.460579 9329 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } }
I20260807 08:29:51.461115 9323 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:51.462060 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba88bf3ec5a64ed28530074969355316. Latest consensus state: current_term: 1 leader_uuid: "ba88bf3ec5a64ed28530074969355316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } } }
I20260807 08:29:51.462174 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:51.462260 9331 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba88bf3ec5a64ed28530074969355316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba88bf3ec5a64ed28530074969355316" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41275 } } }
I20260807 08:29:51.462476 9331 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:51.462684 9345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:51.465479 9345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:51.470528 9345 catalog_manager.cc:1347] Generated new cluster ID: 675430c6f1d64ba4a1c32b664c3c0e7c
I20260807 08:29:51.470587 9345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:51.481177 9345 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:51.482244 9345 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:51.491154 9345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba88bf3ec5a64ed28530074969355316: Generated new TSK 0
I20260807 08:29:51.491689 9345 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:29:51.520388 9365 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 08:29:51.520565 9365 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:29:51.521003 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.503639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.1.157.126:41275)
0807 08:29:51.503752 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.503757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.503923 (+ 166us) server_negotiation.cc:407] Connection header received
0807 08:29:51.504087 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.504090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.504141 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.504238 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:29:51.504452 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.505250 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.506042 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.506417 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.506516 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.506785 (+ 269us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:29:51.520672 (+ 13887us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:29:51.520884 (+ 212us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 08:29:51.521070 9327 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 08:29:51.522257 9364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:51.503552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:41275 (local address 127.0.0.1:59694)
0807 08:29:51.503854 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:51.503866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:51.503962 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:51.504254 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:51.504256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:51.504262 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:29:51.504363 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:51.504368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.505339 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.505341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:51.505949 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:51.505956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.506516 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.506519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:51.506584 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.506624 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:29:51.520854 (+ 14230us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 08:29:51.521008 (+ 154us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.157.126:41275: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.157.126:41275: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41013/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 08:29:51.522943 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9225
2026-08-07T08:29:51Z chronyd exiting
I20260807 08:29:51.566403 1653 test_util.cc:175] -----------------------------------------------
I20260807 08:29:51.566475 1653 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786091364559477-1653-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 08:29:51.567512 1653 mini_oidc.cc:152] Starting JWKS server
I20260807 08:29:51.568312 1653 webserver.cc:460] Webserver started at https://127.0.0.1:37843/ using document root <none> and password file <none>
I20260807 08:29:51.568400 1653 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:29:51.568655 1653 webserver.cc:460] Webserver started at http://127.0.0.1:41377/ using document root <none> and password file <none>
2026-08-07T08:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:51Z Disabled control of system clock
I20260807 08:29:51.577735 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35221
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:35221
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37843/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/testchainca.cert with env {}
W20260807 08:29:51.681762 9374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:51.681972 9374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:51.682008 9374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:51.685827 9374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:51.685866 9374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:51.685880 9374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:51.685894 9374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:51.685907 9374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:29:51.685927 9374 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:29:51.685940 9374 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37843/jwks.json
I20260807 08:29:51.690822 9374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:35221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37843/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:51.691602 9374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:51.692400 9374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:51.699795 9381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:51.699841 9374 server_base.cc:1034] running on GCE node
W20260807 08:29:51.699867 9384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:51.699800 9382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:51.700465 9374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:51.701418 9374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:51.702574 9374 hybrid_clock.cc:648] HybridClock initialized: now 1786091391702557 us; error 29 us; skew 500 ppm
I20260807 08:29:51.704128 9374 webserver.cc:460] Webserver started at http://127.1.157.126:33365/ using document root <none> and password file <none>
I20260807 08:29:51.704627 9374 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:51.704692 9374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:51.704856 9374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:51.706691 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "69661de5d8f44503a68245c5a037edfe"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "4372a83a8891e73cb01067f6d9b50149"
server_key_iv: "6c80c77e77fe100ad8ae08cfed4d016f"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.707194 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "69661de5d8f44503a68245c5a037edfe"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "4372a83a8891e73cb01067f6d9b50149"
server_key_iv: "6c80c77e77fe100ad8ae08cfed4d016f"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.710333 9374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:51.712450 9394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:51.713485 9374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:51.713590 9374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "69661de5d8f44503a68245c5a037edfe"
format_stamp: "Formatted at 2026-08-07 08:29:51 on dist-test-slave-6zbq"
server_key: "4372a83a8891e73cb01067f6d9b50149"
server_key_iv: "6c80c77e77fe100ad8ae08cfed4d016f"
server_key_version: "encryptionkey@0"
I20260807 08:29:51.713678 9374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:51.725037 9374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:51.725556 9374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:51.725705 9374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:51.731855 9374 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:35221
I20260807 08:29:51.731877 9466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:35221 every 8 connection(s)
I20260807 08:29:51.732666 9374 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:51.735419 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:51.740219 9467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe: Bootstrap starting.
I20260807 08:29:51.742298 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9374
I20260807 08:29:51.742425 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:51.742411 9467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:51.742661 1653 external_mini_cluster.cc:1428] Setting key 69588210a2bbcd169a3a4ddcf39f2b63
I20260807 08:29:51.743157 9467 log.cc:824] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:51.745011 9467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe: No bootstrap required, opened a new log
I20260807 08:29:51.748200 9467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } }
I20260807 08:29:51.748390 9467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:51.748495 9467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69661de5d8f44503a68245c5a037edfe, State: Initialized, Role: FOLLOWER
I20260807 08:29:51.748907 9467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } }
I20260807 08:29:51.749053 9467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:51.749115 9467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:51.749218 9467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:51.750154 9467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } }
I20260807 08:29:51.750505 9467 leader_election.cc:302] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69661de5d8f44503a68245c5a037edfe; no voters:
I20260807 08:29:51.750759 9467 leader_election.cc:288] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:51.750756 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.743600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.1.157.126:35221)
0807 08:29:51.744182 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.744196 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.744227 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:29:51.744806 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.744822 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.745114 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.745430 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:51.745819 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.746792 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.747645 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.748087 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.748240 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.748982 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:51.748987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:51.748998 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:51.749421 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:51.749476 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:51.749496 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:51.749568 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:51.749721 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 08:29:51.749951 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
I20260807 08:29:51.750864 9475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:51.751075 9475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [term 1 LEADER]: Becoming Leader. State: Replica: 69661de5d8f44503a68245c5a037edfe, State: Running, Role: LEADER
I20260807 08:29:51.751407 9475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } }
I20260807 08:29:51.751818 9467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:51.752799 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69661de5d8f44503a68245c5a037edfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } } }
I20260807 08:29:51.752933 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:51.753196 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69661de5d8f44503a68245c5a037edfe. Latest consensus state: current_term: 1 leader_uuid: "69661de5d8f44503a68245c5a037edfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69661de5d8f44503a68245c5a037edfe" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35221 } } }
I20260807 08:29:51.753288 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:51.753716 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:29:51.754266 9486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:51.756584 9486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:51.761510 9486 catalog_manager.cc:1347] Generated new cluster ID: b97aa0a639a64e73b8f59c392d5880d7
I20260807 08:29:51.761569 9486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:51.788780 9486 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:51.789865 9486 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:51.800192 9486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69661de5d8f44503a68245c5a037edfe: Generated new TSK 0
I20260807 08:29:51.800843 9486 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 08:29:51.826373 9506 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 08:29:51.826445 9506 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 08:29:51.826936 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:51.809952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.1.157.126:35221)
0807 08:29:51.810061 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:51.810065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:51.810231 (+ 166us) server_negotiation.cc:407] Connection header received
0807 08:29:51.810401 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:51.810404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:51.810447 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:51.810525 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:29:51.810746 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.811243 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.812426 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.812639 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.812704 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.812983 (+ 279us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:29:51.826659 (+ 13676us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:29:51.826843 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35288: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 08:29:51.826982 9474 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35288: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 08:29:51.827133 9504 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:51.809865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35221 (local address 127.0.0.1:35288)
0807 08:29:51.810159 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:51.810173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:51.810281 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:51.810540 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:51.810543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:51.810551 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:29:51.810665 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:51.810672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.811330 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.811334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:51.812329 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:51.812339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:51.812730 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:51.812734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:51.812825 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:51.812853 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:29:51.826806 (+ 13953us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 08:29:51.826956 (+ 150us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.157.126:35221: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
I20260807 08:29:51.827767 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9374
2026-08-07T08:29:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 08:29:51.876603 1653 mini_oidc.cc:152] Starting JWKS server
I20260807 08:29:51.877389 1653 webserver.cc:460] Webserver started at https://127.0.0.1:35521/ using document root <none> and password file <none>
I20260807 08:29:51.877476 1653 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 08:29:51.877725 1653 webserver.cc:460] Webserver started at http://127.0.0.1:39987/ using document root <none> and password file <none>
2026-08-07T08:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:51Z Disabled control of system clock
I20260807 08:29:51.886489 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46233
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:46233
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35521/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:29:51.993150 9519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:51.993484 9519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:51.993569 9519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:51.998050 9519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:51.998116 9519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:51.998155 9519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:51.998186 9519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:51.998215 9519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:29:51.998238 9519 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:29:51.998272 9519 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35521/jwks.json
I20260807 08:29:52.004424 9519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:46233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35521/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:52.005554 9519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:52.007059 9519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:52.016307 9524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:52.016299 9525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:52.016526 9519 server_base.cc:1034] running on GCE node
W20260807 08:29:52.016745 9527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:52.017359 9519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:52.018744 9519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:52.019905 9519 hybrid_clock.cc:648] HybridClock initialized: now 1786091392019879 us; error 42 us; skew 500 ppm
I20260807 08:29:52.021914 9519 webserver.cc:460] Webserver started at http://127.1.157.126:40687/ using document root <none> and password file <none>
I20260807 08:29:52.022518 9519 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:52.022580 9519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:52.022787 9519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:52.024730 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "25937bd94439471a803a15d397b942f4"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "f506ce685a815404bf2bc1f8db91dd2f"
server_key_iv: "16a3283b145ae6cb9229ba847e6cd0a1"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.025283 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "25937bd94439471a803a15d397b942f4"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "f506ce685a815404bf2bc1f8db91dd2f"
server_key_iv: "16a3283b145ae6cb9229ba847e6cd0a1"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.028987 9519 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 08:29:52.031468 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:52.032557 9519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:52.032665 9519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "25937bd94439471a803a15d397b942f4"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "f506ce685a815404bf2bc1f8db91dd2f"
server_key_iv: "16a3283b145ae6cb9229ba847e6cd0a1"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.032764 9519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:52.043697 9519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:52.044231 9519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:52.044375 9519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:52.050457 9519 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:46233
I20260807 08:29:52.050468 9613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:46233 every 8 connection(s)
I20260807 08:29:52.051252 9519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:52.054061 9614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:52.059072 9614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4: Bootstrap starting.
I20260807 08:29:52.061158 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9519
I20260807 08:29:52.061278 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:52.061353 9614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:52.061576 1653 external_mini_cluster.cc:1428] Setting key df2ce44270ab7e2e9501ebd2f1bbf705
I20260807 08:29:52.062151 9614 log.cc:824] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:52.063961 9614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4: No bootstrap required, opened a new log
I20260807 08:29:52.067798 9614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } }
I20260807 08:29:52.068058 9614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:52.068123 9614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25937bd94439471a803a15d397b942f4, State: Initialized, Role: FOLLOWER
I20260807 08:29:52.068532 9614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } }
I20260807 08:29:52.068667 9614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:52.068737 9614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:52.068846 9614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:52.069633 9614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } }
I20260807 08:29:52.069720 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:52.062580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.1.157.126:46233)
0807 08:29:52.063125 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:52.063140 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:52.063173 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:29:52.063754 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:52.063771 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:52.064063 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:52.064338 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:52.064693 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.065582 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.066816 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.067250 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.067358 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.068103 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:52.068109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:52.068125 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:52.068490 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:52.068542 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:52.068561 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:52.068633 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:52.068802 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:29:52.069061 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260807 08:29:52.069952 9614 leader_election.cc:302] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25937bd94439471a803a15d397b942f4; no voters:
I20260807 08:29:52.070230 9614 leader_election.cc:288] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:52.070303 9627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:52.070513 9627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [term 1 LEADER]: Becoming Leader. State: Replica: 25937bd94439471a803a15d397b942f4, State: Running, Role: LEADER
I20260807 08:29:52.070792 9627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } }
I20260807 08:29:52.071381 9614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:52.071775 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25937bd94439471a803a15d397b942f4. Latest consensus state: current_term: 1 leader_uuid: "25937bd94439471a803a15d397b942f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } } }
I20260807 08:29:52.071872 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:52.072144 9629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25937bd94439471a803a15d397b942f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25937bd94439471a803a15d397b942f4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46233 } } }
I20260807 08:29:52.072216 9632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:52.072249 9629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:52.073217 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:29:52.075016 9632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:52.079726 9632 catalog_manager.cc:1347] Generated new cluster ID: 6f29e52fda46461885e8563d32bccfc4
I20260807 08:29:52.079774 9632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:52.084542 9632 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:52.085292 9632 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:52.092149 9632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25937bd94439471a803a15d397b942f4: Generated new TSK 0
I20260807 08:29:52.092667 9632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:29:52.106268 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:52.101450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.1.157.126:46233)
0807 08:29:52.101560 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:52.101564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:52.101739 (+ 175us) server_negotiation.cc:407] Connection header received
0807 08:29:52.101881 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:52.101884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:52.101925 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:52.101999 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:29:52.102220 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.103052 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.104255 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.104454 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.104520 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.104796 (+ 276us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:29:52.105950 (+ 1154us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35521/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:29:52.106154 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35521/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 08:29:52.106323 9623 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35521/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 08:29:52.108206 9660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:52.101370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46233 (local address 127.0.0.1:44984)
0807 08:29:52.101645 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:52.101660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:52.101782 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:52.102012 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:52.102015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:52.102022 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:29:52.102138 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:52.102144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.103171 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.103175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:52.104154 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:52.104165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.104549 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.104553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:52.104643 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.104671 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:29:52.106096 (+ 1425us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35521/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 08:29:52.108030 (+ 1934us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.157.126:46233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35521/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":1739,"thread_start_us":75,"threads_started":1}
I20260807 08:29:52.108736 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9519
E20260807 08:29:52.126122 9661 webserver.cc:533] Webserver: sslize failed:
E20260807 08:29:52.126196 9661 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T08:29:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (293 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T08:29:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:52Z Disabled control of system clock
I20260807 08:29:52.179069 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34223
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:41177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:34223
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/testchainca.cert with env {}
W20260807 08:29:52.283469 9666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:52.283682 9666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:52.283720 9666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:52.287690 9666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:52.287739 9666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:52.287753 9666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:52.287768 9666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:52.287786 9666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 08:29:52.287813 9666 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 08:29:52.287829 9666 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 08:29:52.292853 9666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:34223
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:52.293731 9666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:52.294708 9666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:52.302321 9678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:52.302423 9666 server_base.cc:1034] running on GCE node
W20260807 08:29:52.302326 9676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:52.302333 9674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:52.303056 9666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:52.304021 9666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:52.305186 9666 hybrid_clock.cc:648] HybridClock initialized: now 1786091392305173 us; error 28 us; skew 500 ppm
I20260807 08:29:52.306944 9666 webserver.cc:460] Webserver started at http://127.1.157.126:46425/ using document root <none> and password file <none>
I20260807 08:29:52.307451 9666 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:52.307515 9666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:52.307684 9666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:52.309559 9666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "2642d4abda5b4171b09d42463ac52566"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "5cac7605c77243114ac4cce48836acd9"
server_key_iv: "37bdc35f135f62fc3c5b696cc1bb02df"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.310078 9666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "2642d4abda5b4171b09d42463ac52566"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "5cac7605c77243114ac4cce48836acd9"
server_key_iv: "37bdc35f135f62fc3c5b696cc1bb02df"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.313390 9666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:29:52.315585 9685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:52.316653 9666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:52.316762 9666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "2642d4abda5b4171b09d42463ac52566"
format_stamp: "Formatted at 2026-08-07 08:29:52 on dist-test-slave-6zbq"
server_key: "5cac7605c77243114ac4cce48836acd9"
server_key_iv: "37bdc35f135f62fc3c5b696cc1bb02df"
server_key_version: "encryptionkey@0"
I20260807 08:29:52.316851 9666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:52.330157 9666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:52.330739 9666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:52.330890 9666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:52.337268 9666 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:34223
I20260807 08:29:52.337285 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:34223 every 8 connection(s)
I20260807 08:29:52.338085 9666 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:52.341027 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:52.343892 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9666
I20260807 08:29:52.344010 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:52.344300 1653 external_mini_cluster.cc:1428] Setting key 76865c2fed58693b60eee6cea21c86f3
I20260807 08:29:52.346777 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566: Bootstrap starting.
I20260807 08:29:52.348886 9757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:52.349565 9757 log.cc:824] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:52.351531 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566: No bootstrap required, opened a new log
I20260807 08:29:52.353296 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:52.345643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.1.157.126:34223)
0807 08:29:52.346244 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:52.346269 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:52.346303 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:29:52.346965 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:52.346984 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:52.347364 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:52.347653 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:52.348065 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.349043 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.349911 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.350531 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.350682 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.351389 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:52.351395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:52.351410 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:29:52.351867 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:52.351939 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:29:52.351965 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:52.352061 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:52.352181 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:29:52.352441 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260807 08:29:52.355611 9757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } }
I20260807 08:29:52.355880 9757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:52.355938 9757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2642d4abda5b4171b09d42463ac52566, State: Initialized, Role: FOLLOWER
I20260807 08:29:52.356052 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:29:52.356331 9757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } }
I20260807 08:29:52.356460 9757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:52.356524 9757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:52.356614 9757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:52.357595 9757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } }
I20260807 08:29:52.357918 9757 leader_election.cc:302] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2642d4abda5b4171b09d42463ac52566; no voters:
I20260807 08:29:52.358177 9757 leader_election.cc:288] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:52.358281 9763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:52.358539 9763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [term 1 LEADER]: Becoming Leader. State: Replica: 2642d4abda5b4171b09d42463ac52566, State: Running, Role: LEADER
I20260807 08:29:52.358796 9763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } }
I20260807 08:29:52.359236 9757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:52.360240 9764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2642d4abda5b4171b09d42463ac52566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } } }
I20260807 08:29:52.360383 9764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:52.360643 9765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2642d4abda5b4171b09d42463ac52566. Latest consensus state: current_term: 1 leader_uuid: "2642d4abda5b4171b09d42463ac52566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2642d4abda5b4171b09d42463ac52566" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34223 } } }
I20260807 08:29:52.360749 9765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:52.360828 9775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:52.363729 9775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:52.368952 9775 catalog_manager.cc:1347] Generated new cluster ID: c40b2f2b58804f5593166c777e89599d
I20260807 08:29:52.369001 9775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:52.396632 9775 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:52.397468 9775 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:52.402295 9775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2642d4abda5b4171b09d42463ac52566: Generated new TSK 0
I20260807 08:29:52.402918 9775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:29:52.853156 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:52.415746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.1.157.126:34223)
0807 08:29:52.415873 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:52.415877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:52.416039 (+ 162us) server_negotiation.cc:407] Connection header received
0807 08:29:52.416193 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:52.416196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:52.416239 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:52.416319 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0807 08:29:52.416552 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.417083 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.418263 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.418510 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.418582 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.418917 (+ 335us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 08:29:52.852781 (+433864us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:29:52.853034 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33980: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 08:29:52.853233 9761 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33980: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 08:29:52.853397 9802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:52.415686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34223 (local address 127.0.0.1:33980)
0807 08:29:52.415957 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:52.415971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:52.416081 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:52.416336 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:52.416339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:52.416348 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 08:29:52.416467 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:52.416474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.417174 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.417178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:52.418157 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:52.418168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:52.418621 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:52.418626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:52.418732 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:52.418775 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 08:29:52.853016 (+434241us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 08:29:52.853196 (+ 180us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.157.126:34223: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260807 08:29:52.854158 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9666
2026-08-07T08:29:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (709 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (118 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (115 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:53 dist-test-slave-6zbq krb5kdc[9823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:53 dist-test-slave-6zbq krb5kdc[9823](info): set up 2 sockets
Aug 07 08:29:53 dist-test-slave-6zbq krb5kdc[9823](info): commencing operation
krb5kdc: starting...
W20260807 08:29:54.216363 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:29:54.326581 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39217
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:39217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:54.432222 9839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:54.432427 9839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:54.432466 9839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:54.436302 9839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:54.436348 9839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:54.436379 9839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:54.436409 9839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:54.436439 9839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:54.441402 9839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:39217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:54.442230 9839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:54.443296 9839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:54.450846 9849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:54.450941 9839 server_base.cc:1034] running on GCE node
W20260807 08:29:54.450841 9853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:54.450902 9850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:54.451702 9839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:54.452670 9839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:54.453828 9839 hybrid_clock.cc:648] HybridClock initialized: now 1786091394453808 us; error 32 us; skew 500 ppm
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:54.460685 9839 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:29:54.461544 9839 webserver.cc:460] Webserver started at http://127.1.157.126:42399/ using document root <none> and password file <none>
I20260807 08:29:54.462028 9839 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:54.462093 9839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:54.462309 9839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:54.464254 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "91578fa54ab447cb8b9de17637a97c73"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "e5478bd9ac461aeab086b430a4433270"
server_key_iv: "c873431d7c21c0745f2551b04a7c14a3"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.464774 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "91578fa54ab447cb8b9de17637a97c73"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "e5478bd9ac461aeab086b430a4433270"
server_key_iv: "c873431d7c21c0745f2551b04a7c14a3"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.468205 9839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 08:29:54.470290 9861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.471441 9839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:54.471544 9839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "91578fa54ab447cb8b9de17637a97c73"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "e5478bd9ac461aeab086b430a4433270"
server_key_iv: "c873431d7c21c0745f2551b04a7c14a3"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.471627 9839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:54.494702 9839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:54.497431 9839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:54.497586 9839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:54.504204 9839 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:39217
I20260807 08:29:54.504225 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:39217 every 8 connection(s)
I20260807 08:29:54.505030 9839 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:54.507586 9933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:54.512012 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9839
I20260807 08:29:54.512131 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:54.512387 1653 external_mini_cluster.cc:1428] Setting key cf6da1f3866c30c09aac9e1a8e69185a
I20260807 08:29:54.512809 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73: Bootstrap starting.
I20260807 08:29:54.514932 9933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:54.515731 9933 log.cc:824] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:54.517529 9933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73: No bootstrap required, opened a new log
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:54.521574 9933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } }
I20260807 08:29:54.521771 9933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:54.521816 9933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91578fa54ab447cb8b9de17637a97c73, State: Initialized, Role: FOLLOWER
I20260807 08:29:54.522233 9933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } }
I20260807 08:29:54.522398 9933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:54.522459 9933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:54.522559 9933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:54.523535 9933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } }
I20260807 08:29:54.523844 9933 leader_election.cc:302] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91578fa54ab447cb8b9de17637a97c73; no voters:
I20260807 08:29:54.524192 9933 leader_election.cc:288] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:54.524286 9943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:54.524492 9943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [term 1 LEADER]: Becoming Leader. State: Replica: 91578fa54ab447cb8b9de17637a97c73, State: Running, Role: LEADER
I20260807 08:29:54.524783 9943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } }
I20260807 08:29:54.525247 9933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:54.526392 9946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91578fa54ab447cb8b9de17637a97c73. Latest consensus state: current_term: 1 leader_uuid: "91578fa54ab447cb8b9de17637a97c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } } }
I20260807 08:29:54.526515 9946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:54.526832 9945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91578fa54ab447cb8b9de17637a97c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91578fa54ab447cb8b9de17637a97c73" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 39217 } } }
I20260807 08:29:54.526921 9945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:54.527050 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:54.513637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.1.157.126:39217)
0807 08:29:54.514011 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:54.514025 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:54.514059 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:29:54.514717 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:54.514736 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:54.515048 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:54.515305 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:54.516202 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.517195 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.518316 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.518816 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.518939 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:54.521565 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:54.521570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:54.521582 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:54.521607 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:54.523483 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:54.523890 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:54.523896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:54.523901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:54.524039 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:54.524343 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:54.524346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:54.524348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:54.524836 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:54.525664 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:54.525815 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 08:29:54.526098 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260807 08:29:54.527505 9958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:54.530217 9958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:54.535178 9958 catalog_manager.cc:1347] Generated new cluster ID: 7de4ead2740845fe9c34c5c301fc2193
I20260807 08:29:54.535238 9958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:54.555243 9958 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:54.556028 9958 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:54.562352 9958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91578fa54ab447cb8b9de17637a97c73: Generated new TSK 0
I20260807 08:29:54.562937 9958 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:29:54.592693 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:54.727864 9971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:54.728077 9971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:54.728116 9971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:54.731905 9971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:54.731949 9971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:54.732002 9971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:54.736804 9971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:54.737612 9971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:54.738513 9971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:54.746961 9982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:54.746954 9979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:54.746953 9978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:54.747191 9971 server_base.cc:1034] running on GCE node
I20260807 08:29:54.747586 9971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:54.748134 9971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:54.749310 9971 hybrid_clock.cc:648] HybridClock initialized: now 1786091394749300 us; error 27 us; skew 500 ppm
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:54.751837 9971 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:29:54.752626 9971 webserver.cc:460] Webserver started at http://127.1.157.65:38949/ using document root <none> and password file <none>
I20260807 08:29:54.753099 9971 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:54.753163 9971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:54.753341 9971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:54.755184 9971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "65d38cf1ff3e491fadb10c9cf82ddb8f"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "3e1f3a65d7252f2d2c2fa7741e09faf3"
server_key_iv: "b3df7201bf728858300202dfb332f289"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.755693 9971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "65d38cf1ff3e491fadb10c9cf82ddb8f"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "3e1f3a65d7252f2d2c2fa7741e09faf3"
server_key_iv: "b3df7201bf728858300202dfb332f289"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.758937 9971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:54.761036 9992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.762082 9971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:54.762184 9971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "65d38cf1ff3e491fadb10c9cf82ddb8f"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "3e1f3a65d7252f2d2c2fa7741e09faf3"
server_key_iv: "b3df7201bf728858300202dfb332f289"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.762270 9971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:54.773269 9971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:54.776090 9971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:54.776248 9971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:54.776758 9971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:54.777480 9971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:54.777527 9971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.777576 9971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:54.777606 9971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.785916 9971 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:35831
I20260807 08:29:54.785949 10159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:35831 every 8 connection(s)
I20260807 08:29:54.786746 9971 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:54.788460 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 9971
I20260807 08:29:54.788578 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:54.788894 1653 external_mini_cluster.cc:1428] Setting key 1435104ffd0f050706058d5e3423d0d9
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:54.799916 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:54.788333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:56995 (local address 127.1.157.126:39217)
0807 08:29:54.788443 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:54.788447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:54.789446 (+ 999us) server_negotiation.cc:407] Connection header received
0807 08:29:54.790974 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:54.790978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:54.791027 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:54.791116 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:54.792198 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.792727 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.793824 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.794059 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.794147 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:54.796786 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:54.796789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:54.796791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:54.796818 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:54.798067 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:54.798649 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:54.798651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:54.798653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:54.798732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:54.798997 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:54.798999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:54.799000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:54.799195 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:54.799286 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:54.799700 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 08:29:54.799797 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:54.800727 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:54.788795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39217 (local address 127.1.157.65:56995)
0807 08:29:54.789273 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:54.789329 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:54.790769 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:54.791248 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:54.791253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:54.791582 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:54.792055 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:54.792068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.792855 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.792858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:54.793704 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:54.793714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:54.794197 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:54.794201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:54.794405 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:54.794916 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:54.794932 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:54.796589 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:54.798185 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:54.798192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:54.798205 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:54.798562 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:54.798803 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:54.798805 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:54.798806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:54.798928 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:54.799293 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:54.799297 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:54.799510 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 08:29:54.799650 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 08:29:54.799872 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
I20260807 08:29:54.801841 10161 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39217
I20260807 08:29:54.802073 10161 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:54.802646 10161 heartbeater.cc:511] Master 127.1.157.126:39217 requested a full tablet report, sending...
I20260807 08:29:54.804102 9887 ts_manager.cc:194] Registered new tserver with Master: 65d38cf1ff3e491fadb10c9cf82ddb8f (127.1.157.65:35831)
I20260807 08:29:54.805200 9887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:56995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:29:54.814208 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:54.919834 10167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:54.920046 10167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:54.920101 10167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:54.924002 10167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:54.924053 10167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:54.924126 10167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:54.929101 10167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:54.929962 10167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:54.930882 10167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:29:54.939765 10167 server_base.cc:1034] running on GCE node
W20260807 08:29:54.939743 10173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:54.939709 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:54.939718 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:54.940374 10167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:54.940933 10167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:54.942087 10167 hybrid_clock.cc:648] HybridClock initialized: now 1786091394942072 us; error 28 us; skew 500 ppm
Aug 07 08:29:54 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:54.944729 10167 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:29:54.945555 10167 webserver.cc:460] Webserver started at http://127.1.157.66:42995/ using document root <none> and password file <none>
I20260807 08:29:54.946070 10167 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:54.946138 10167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:54.946318 10167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:54.948293 10167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "f5275fd590754fdb8a736737a9bc45eb"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "f7fcdc0d34c582b9bfa670f1ff58b7f7"
server_key_iv: "d40e7039b99d44c70b75ec3538e5b6f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.948822 10167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "f5275fd590754fdb8a736737a9bc45eb"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "f7fcdc0d34c582b9bfa670f1ff58b7f7"
server_key_iv: "d40e7039b99d44c70b75ec3538e5b6f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.952281 10167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:54.954470 10189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.955538 10167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:54.955648 10167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "f5275fd590754fdb8a736737a9bc45eb"
format_stamp: "Formatted at 2026-08-07 08:29:54 on dist-test-slave-6zbq"
server_key: "f7fcdc0d34c582b9bfa670f1ff58b7f7"
server_key_iv: "d40e7039b99d44c70b75ec3538e5b6f8"
server_key_version: "encryptionkey@0"
I20260807 08:29:54.955744 10167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:54.986560 10167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:54.989370 10167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:54.989529 10167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:54.990064 10167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:54.990875 10167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:54.990921 10167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.990962 10167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:54.990991 10167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:54.999068 10167 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:39749
I20260807 08:29:54.999106 10355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:39749 every 8 connection(s)
I20260807 08:29:54.999866 10167 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091394, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:55.009507 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 10167
I20260807 08:29:55.009644 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:55.009887 1653 external_mini_cluster.cc:1428] Setting key ddd6f6271eefa893958c5adbd5729ddd
I20260807 08:29:55.012144 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.001458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:52599 (local address 127.1.157.126:39217)
0807 08:29:55.001594 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.001599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.002375 (+ 776us) server_negotiation.cc:407] Connection header received
0807 08:29:55.003639 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.003642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.003687 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.003757 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.004770 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.005274 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.006323 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.006578 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.006652 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.008763 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:55.008766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:55.008768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:55.008794 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:55.010140 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.010699 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.010702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.010704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.010796 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.011101 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.011104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.011106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.011333 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:55.011412 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:55.011939 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 08:29:55.012032 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:29:55.013095 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.001803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39217 (local address 127.1.157.66:52599)
0807 08:29:55.002229 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:55.002268 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:55.003468 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:55.003894 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:55.003900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:55.004221 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:55.004657 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:55.004670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.005368 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.005370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:55.006229 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:55.006238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.006683 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.006685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:55.006820 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.007240 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:55.007255 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:55.008603 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:55.010247 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:55.010252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:55.010261 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:55.010608 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:55.010880 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:55.010882 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:55.010884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:55.011006 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:55.011438 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:55.011444 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:55.011715 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 08:29:55.011903 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 08:29:55.012244 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":115,"threads_started":1}
I20260807 08:29:55.014225 10356 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39217
I20260807 08:29:55.014446 10356 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:55.014869 10356 heartbeater.cc:511] Master 127.1.157.126:39217 requested a full tablet report, sending...
I20260807 08:29:55.015799 9887 ts_manager.cc:194] Registered new tserver with Master: f5275fd590754fdb8a736737a9bc45eb (127.1.157.66:39749)
I20260807 08:29:55.016341 9887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:52599
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:29:55.034528 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:55.138928 10361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:55.139123 10361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:55.139160 10361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:55.143024 10361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:55.143067 10361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:55.143122 10361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:55.148021 10361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:55.148821 10361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:55.149667 10361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:29:55.158107 10361 server_base.cc:1034] running on GCE node
W20260807 08:29:55.158044 10368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:55.158054 10367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:55.158054 10371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:55.158749 10361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:55.159343 10361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:55.160502 10361 hybrid_clock.cc:648] HybridClock initialized: now 1786091395160481 us; error 31 us; skew 500 ppm
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.162956 10361 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:29:55.163699 10361 webserver.cc:460] Webserver started at http://127.1.157.67:46667/ using document root <none> and password file <none>
I20260807 08:29:55.164165 10361 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:55.164222 10361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:55.164412 10361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:55.166364 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "2812aefa49a34dbfb6fca8634abb6b0d"
format_stamp: "Formatted at 2026-08-07 08:29:55 on dist-test-slave-6zbq"
server_key: "6d2b31caa34f84953049f1aa15e822d1"
server_key_iv: "7e8983a35939267ff7f0c92275685f05"
server_key_version: "encryptionkey@0"
I20260807 08:29:55.166896 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "2812aefa49a34dbfb6fca8634abb6b0d"
format_stamp: "Formatted at 2026-08-07 08:29:55 on dist-test-slave-6zbq"
server_key: "6d2b31caa34f84953049f1aa15e822d1"
server_key_iv: "7e8983a35939267ff7f0c92275685f05"
server_key_version: "encryptionkey@0"
I20260807 08:29:55.170118 10361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:29:55.172529 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:55.173621 10361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:29:55.173723 10361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "2812aefa49a34dbfb6fca8634abb6b0d"
format_stamp: "Formatted at 2026-08-07 08:29:55 on dist-test-slave-6zbq"
server_key: "6d2b31caa34f84953049f1aa15e822d1"
server_key_iv: "7e8983a35939267ff7f0c92275685f05"
server_key_version: "encryptionkey@0"
I20260807 08:29:55.173817 10361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:55.189483 10361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:55.192190 10361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:55.192350 10361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:55.192837 10361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:55.193554 10361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:55.193599 10361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:55.193646 10361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:55.193673 10361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:55.201625 10361 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:38861
I20260807 08:29:55.201682 10544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:38861 every 8 connection(s)
I20260807 08:29:55.202406 10361 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:55.209699 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 10361
I20260807 08:29:55.209836 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:55.210120 1653 external_mini_cluster.cc:1428] Setting key 47011be08965aebf1a63db803fc208fb
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:29:55.214701 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.203931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:50419 (local address 127.1.157.126:39217)
0807 08:29:55.204061 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.204066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.204831 (+ 765us) server_negotiation.cc:407] Connection header received
0807 08:29:55.206104 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.206107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.206154 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.206225 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.207237 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.207753 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.208785 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.208990 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.209057 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.211452 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:55.211455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:55.211458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:55.211488 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:55.212800 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.213259 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.213263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.213264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.213357 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.213660 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.213664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.213666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.213847 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:55.213933 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:55.214417 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 08:29:55.214544 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:29:55.215492 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.204286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:39217 (local address 127.1.157.67:50419)
0807 08:29:55.204693 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:55.204731 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:55.205922 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:55.206401 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:55.206408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:55.206712 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:55.207126 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:55.207138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.207841 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.207843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:55.208692 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:55.208701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.209088 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.209090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:55.209214 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.209670 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:55.209690 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:55.211295 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:55.212913 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:55.212918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:55.212932 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:55.213165 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:55.213439 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:55.213442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:55.213445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:55.213573 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:55.213946 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:55.213953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:55.214167 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 08:29:55.214325 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 08:29:55.214673 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
I20260807 08:29:55.216490 10545 heartbeater.cc:347] Connected to a master server at 127.1.157.126:39217
I20260807 08:29:55.216703 10545 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:55.217145 10545 heartbeater.cc:511] Master 127.1.157.126:39217 requested a full tablet report, sending...
I20260807 08:29:55.218032 9887 ts_manager.cc:194] Registered new tserver with Master: 2812aefa49a34dbfb6fca8634abb6b0d (127.1.157.67:38861)
I20260807 08:29:55.218595 9887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:50419
I20260807 08:29:55.223681 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/test-admin.keytab.
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.238162 1653 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 08:29:55.240593 10556 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:55.239370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48660 (local address 127.1.157.65:35831)
0807 08:29:55.239694 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.239702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.239723 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:29:55.239774 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.239779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.239921 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.240025 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.240423 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48660: BlockingRecv error: recv got EOF from 127.0.0.1:48660 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":125,"threads_started":1}
W20260807 08:29:55.240653 1653 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.241573 1653 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.242419 1653 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:55.251145 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.243187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.1.157.65:35831)
0807 08:29:55.243294 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.243298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.243359 (+ 61us) server_negotiation.cc:407] Connection header received
0807 08:29:55.243481 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.243483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.243520 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.243580 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.244077 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.244823 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.245607 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.245946 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.246040 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.248045 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:55.248054 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:55.248064 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:55.248105 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:55.249450 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.249854 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.249860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.249862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.249937 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.250182 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.250185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.250187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.250468 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:55.250680 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:55.250869 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 08:29:55.251012 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 08:29:55.254215 10556 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:55.253467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.1.157.65:35831)
0807 08:29:55.253560 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.253563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.253579 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:29:55.253675 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.253677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.253709 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.253776 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.254147 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48684: BlockingRecv error: recv got EOF from 127.0.0.1:48684 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260807 08:29:55.254400 1653 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.255237 1653 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.255975 1653 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:55.263489 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.256723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.1.157.65:35831)
0807 08:29:55.256820 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.256823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.256854 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:29:55.256947 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.256949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.256979 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.257037 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.257425 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.257907 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.258705 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.258926 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.259012 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.261023 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:55.261026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:55.261027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:55.261048 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:55.262189 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.262600 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.262603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.262604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.262689 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.262874 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.262876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.262877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.263012 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:55.263097 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:55.263307 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 08:29:55.263393 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 08:29:55.265614 10556 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:29:55.264837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.1.157.65:35831)
0807 08:29:55.264951 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.264953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.264984 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:29:55.265084 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.265086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.265124 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.265181 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.265529 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48698: BlockingRecv error: recv got EOF from 127.0.0.1:48698 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 08:29:55.265688 1653 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786091364559477-1653-0/krb5kdc/krb5cc' of type FILE
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.266613 1653 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:55.267380 1653 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[9823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:29:55.274989 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:55.268035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.1.157.65:35831)
0807 08:29:55.268143 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:55.268147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:55.268244 (+ 97us) server_negotiation.cc:407] Connection header received
0807 08:29:55.268344 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:55.268347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:55.268382 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:55.268440 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:55.268906 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.269354 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.270135 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:55.270378 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:55.270473 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:55.272457 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:55.272461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:55.272464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:55.272485 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:55.273492 (+ 1007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.273871 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.273875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.273877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.273949 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:55.274189 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:55.274192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:55.274194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:55.274410 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:55.274523 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:55.274752 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 08:29:55.274859 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:29:55.275890 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9971
I20260807 08:29:55.288638 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 10167
I20260807 08:29:55.298887 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 10361
I20260807 08:29:55.309772 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 9839
2026-08-07T08:29:55Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2218 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[10573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[10573](info): set up 2 sockets
Aug 07 08:29:55 dist-test-slave-6zbq krb5kdc[10573](info): commencing operation
krb5kdc: starting...
W20260807 08:29:56.425120 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:29:56 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:29:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:29:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab.
Entry for principal oryx/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab.
I20260807 08:29:56.535076 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:38219
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab
--principal=oryx/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:38219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:56.642623 10595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:56.642844 10595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:56.642897 10595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:56.646756 10595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:56.646801 10595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:56.646828 10595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:56.646850 10595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:56.646879 10595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:56.651799 10595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:38219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab
--principal=oryx/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:38219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:56.652665 10595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:56.653545 10595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:56.661161 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:56.661278 10595 server_base.cc:1034] running on GCE node
W20260807 08:29:56.661167 10605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:56.661168 10602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:56.661936 10595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:56.662933 10595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:56.664089 10595 hybrid_clock.cc:648] HybridClock initialized: now 1786091396664062 us; error 41 us; skew 500 ppm
Aug 07 08:29:56 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:56.666571 10595 init.cc:377] Logged in from keytab as oryx/127.1.157.126@KRBTEST.COM (short username oryx)
I20260807 08:29:56.667331 10595 webserver.cc:460] Webserver started at http://127.1.157.126:33881/ using document root <none> and password file <none>
I20260807 08:29:56.667812 10595 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:56.667874 10595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:56.668044 10595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:56.669924 10595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "38352ac8d21c448f974575b7dc9f7d54"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "b674d0e96f4433324e8db84c29563eb0"
server_key_iv: "05da1eb23f330e9fe20402d654a72382"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.670467 10595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "38352ac8d21c448f974575b7dc9f7d54"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "b674d0e96f4433324e8db84c29563eb0"
server_key_iv: "05da1eb23f330e9fe20402d654a72382"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.673826 10595 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:29:56.676018 10616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:56.677073 10595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:29:56.677181 10595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "38352ac8d21c448f974575b7dc9f7d54"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "b674d0e96f4433324e8db84c29563eb0"
server_key_iv: "05da1eb23f330e9fe20402d654a72382"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.677269 10595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:56.688087 10595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:56.690855 10595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:56.691010 10595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:56.697350 10595 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:38219
I20260807 08:29:56.697584 10696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:38219 every 8 connection(s)
I20260807 08:29:56.698167 10595 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:29:56.699961 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 10595
I20260807 08:29:56.700064 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:29:56.700318 1653 external_mini_cluster.cc:1428] Setting key 9c5efac3456e191864a79266037c149a
I20260807 08:29:56.700717 10697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:56.707405 10697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Bootstrap starting.
Aug 07 08:29:56 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:29:56.709738 10697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:56.710587 10697 log.cc:824] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:56.712446 10697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: No bootstrap required, opened a new log
I20260807 08:29:56.714395 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:56.701895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.1.157.126:38219)
0807 08:29:56.702288 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:56.702309 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:56.702374 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:29:56.703241 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:56.703264 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:56.703649 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:56.704001 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:56.704626 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.706104 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.706924 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.707577 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.707733 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:56.710230 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:56.710235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:56.710246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:56.710272 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:56.711847 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:56.712308 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:56.712312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:56.712316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:56.712415 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:56.712695 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:56.712697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:56.712699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:56.713044 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:56.713183 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:56.713458 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 08:29:56.713701 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260807 08:29:56.716179 10697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } }
I20260807 08:29:56.716446 10697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:56.716547 10697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38352ac8d21c448f974575b7dc9f7d54, State: Initialized, Role: FOLLOWER
I20260807 08:29:56.716938 10697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } }
I20260807 08:29:56.717059 10697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:29:56.717119 10697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:29:56.717207 10697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:56.717967 10697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } }
I20260807 08:29:56.718271 10697 leader_election.cc:302] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38352ac8d21c448f974575b7dc9f7d54; no voters:
I20260807 08:29:56.718559 10697 leader_election.cc:288] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:29:56.718648 10702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:56.718853 10702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 LEADER]: Becoming Leader. State: Replica: 38352ac8d21c448f974575b7dc9f7d54, State: Running, Role: LEADER
I20260807 08:29:56.719182 10702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } }
I20260807 08:29:56.719587 10697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:56.720762 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38352ac8d21c448f974575b7dc9f7d54. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } }
I20260807 08:29:56.720903 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:56.720749 10703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } }
I20260807 08:29:56.721213 10703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:56.721328 10711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:29:56.723907 10711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:29:56.728715 10711 catalog_manager.cc:1347] Generated new cluster ID: fc29cca629f44c0cb4b812553eca344c
I20260807 08:29:56.728772 10711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:29:56.744531 10711 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:29:56.745345 10711 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:29:56.752619 10711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Generated new TSK 0
I20260807 08:29:56.753185 10711 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
I20260807 08:29:56.781051 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:56.885963 10727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:56.886183 10727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:56.886221 10727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:56.890172 10727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:56.890233 10727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:56.890291 10727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:56.895259 10727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:56.896154 10727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:56.897128 10727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:56.905891 10734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:56.905966 10727 server_base.cc:1034] running on GCE node
W20260807 08:29:56.905917 10735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:56.905917 10737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:56.906466 10727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:56.907018 10727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:56.908186 10727 hybrid_clock.cc:648] HybridClock initialized: now 1786091396908171 us; error 29 us; skew 500 ppm
Aug 07 08:29:56 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:56.910782 10727 init.cc:377] Logged in from keytab as oryx/127.1.157.65@KRBTEST.COM (short username oryx)
I20260807 08:29:56.911558 10727 webserver.cc:460] Webserver started at http://127.1.157.65:40841/ using document root <none> and password file <none>
I20260807 08:29:56.912040 10727 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:56.912106 10727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:56.912299 10727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:56.914156 10727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "24623ec868a04df390a3223e0c079963"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "ddb241b0c828eb0c16d6f595311e2a1a"
server_key_iv: "139b36f3e6b865df2f3d134cfc0fe4b2"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.914702 10727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "24623ec868a04df390a3223e0c079963"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "ddb241b0c828eb0c16d6f595311e2a1a"
server_key_iv: "139b36f3e6b865df2f3d134cfc0fe4b2"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.918006 10727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:29:56.920256 10748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:56.921505 10727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:56.921628 10727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "24623ec868a04df390a3223e0c079963"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "ddb241b0c828eb0c16d6f595311e2a1a"
server_key_iv: "139b36f3e6b865df2f3d134cfc0fe4b2"
server_key_version: "encryptionkey@0"
I20260807 08:29:56.921717 10727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:56.950920 10727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:56.954154 10727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:56.954372 10727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:56.954968 10727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:56.955765 10727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:56.955812 10727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:56.955865 10727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:56.955891 10727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:56.963932 10727 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:46013
I20260807 08:29:56.963963 10914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:46013 every 8 connection(s)
I20260807 08:29:56.964757 10727 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:56.966544 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 10727
I20260807 08:29:56.966667 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:29:56.966967 1653 external_mini_cluster.cc:1428] Setting key f7986b9ae202c1263cfcdfbf1b340030
Aug 07 08:29:56 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:29:56.978024 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:56.966443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52461 (local address 127.1.157.126:38219)
0807 08:29:56.966562 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:56.966566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:56.968197 (+ 1631us) server_negotiation.cc:407] Connection header received
0807 08:29:56.969141 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:56.969144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:56.969197 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:56.969267 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:56.970415 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.970968 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.972055 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.972290 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.972377 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:56.974974 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:56.974977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:56.974979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:56.975003 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:56.976251 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:56.976802 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:56.976804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:56.976806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:56.976882 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:56.977128 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:56.977130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:56.977131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:56.977304 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:56.977396 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:56.977810 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 08:29:56.977907 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.66@KRBTEST.COM" created.
I20260807 08:29:56.978927 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:56.966914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:52461)
0807 08:29:56.968023 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:56.968077 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:56.968950 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:56.969446 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:56.969452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:56.969787 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:56.970231 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:56.970254 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.971087 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.971090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:56.971939 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:56.971949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:56.972427 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:56.972431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:56.972589 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:56.973121 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:56.973137 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:56.974815 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:56.976371 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:56.976377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:56.976391 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:56.976711 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:56.976946 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:56.976948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:56.976949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:56.977066 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:56.977389 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:56.977393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:56.977609 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 08:29:56.977765 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:29:56.977994 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":720,"threads_started":1}
I20260807 08:29:56.980171 10916 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
I20260807 08:29:56.980391 10916 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:56.980906 10916 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:56.982329 10639 ts_manager.cc:194] Registered new tserver with Master: 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013)
I20260807 08:29:56.983497 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:52461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
I20260807 08:29:56.992491 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:57.097419 10921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:57.097627 10921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:57.097682 10921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:57.101533 10921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:57.101581 10921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:57.101655 10921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:57.106535 10921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:57.107364 10921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:57.108219 10921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:57.116755 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.116792 10921 server_base.cc:1034] running on GCE node
W20260807 08:29:57.116750 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.116945 10933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.117298 10921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:57.117830 10921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:57.118976 10921 hybrid_clock.cc:648] HybridClock initialized: now 1786091397118963 us; error 28 us; skew 500 ppm
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:57.121465 10921 init.cc:377] Logged in from keytab as oryx/127.1.157.66@KRBTEST.COM (short username oryx)
I20260807 08:29:57.122206 10921 webserver.cc:460] Webserver started at http://127.1.157.66:34265/ using document root <none> and password file <none>
I20260807 08:29:57.122710 10921 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.122769 10921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:57.123020 10921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:57.124845 10921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "8f63837b354c40a8afe022d9db8369b2"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "ce7816333a3562a62e5db2f6d6051c4e"
server_key_iv: "a7e596c33e401bc3c04e120a119a684d"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.125371 10921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "8f63837b354c40a8afe022d9db8369b2"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "ce7816333a3562a62e5db2f6d6051c4e"
server_key_iv: "a7e596c33e401bc3c04e120a119a684d"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.128660 10921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:29:57.130733 10945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.131762 10921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:57.131855 10921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "8f63837b354c40a8afe022d9db8369b2"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "ce7816333a3562a62e5db2f6d6051c4e"
server_key_iv: "a7e596c33e401bc3c04e120a119a684d"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.131945 10921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.143958 10921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:57.146239 10921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:57.146445 10921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:57.146906 10921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:57.147589 10921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:57.147632 10921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.147672 10921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:57.147702 10921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.155329 10921 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:36719
I20260807 08:29:57.155364 11106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:36719 every 8 connection(s)
I20260807 08:29:57.156117 10921 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:57.157977 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 10921
I20260807 08:29:57.158079 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:29:57.158413 1653 external_mini_cluster.cc:1428] Setting key e4523c19101f488c047798dcfc2f3664
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:29:57.168031 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.157611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:44421 (local address 127.1.157.126:38219)
0807 08:29:57.157740 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.157744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.158807 (+ 1063us) server_negotiation.cc:407] Connection header received
0807 08:29:57.159699 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.159702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.159761 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.159831 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.160848 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.161369 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.162441 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.162662 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.162757 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.164833 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.164836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.164838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.164860 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.166141 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.166639 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.166642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.166643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.166717 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.167027 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.167029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.167030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.167197 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.167279 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.167762 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.167880 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:29:57.168722 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.157938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:44421)
0807 08:29:57.158653 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.158703 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.159507 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.159992 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.159998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.160313 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.160718 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.160730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.161490 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.161493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.162312 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.162322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.162791 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.162796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.162949 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.163416 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.163431 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.164687 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.166254 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.166260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.166272 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.166549 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.166802 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.166806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.166808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.166934 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.167298 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.167304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.167529 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.167672 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.167948 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":115,"threads_started":1}
I20260807 08:29:57.169683 11107 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.67@KRBTEST.COM" created.
I20260807 08:29:57.169893 11107 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:57.170382 11107 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:57.171263 10639 ts_manager.cc:194] Registered new tserver with Master: 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:57.171790 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:44421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
I20260807 08:29:57.182615 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:57.289660 11112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:57.289903 11112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:57.289943 11112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:57.293823 11112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:57.293911 11112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:57.293970 11112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:57.299404 11112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:38219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:57.300418 11112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:57.301427 11112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:57.310544 11121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.310639 11112 server_base.cc:1034] running on GCE node
W20260807 08:29:57.310554 11122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.310596 11125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.311199 11112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:57.311777 11112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:57.312922 11112 hybrid_clock.cc:648] HybridClock initialized: now 1786091397312898 us; error 36 us; skew 500 ppm
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:57.315667 11112 init.cc:377] Logged in from keytab as oryx/127.1.157.67@KRBTEST.COM (short username oryx)
I20260807 08:29:57.316448 11112 webserver.cc:460] Webserver started at http://127.1.157.67:40703/ using document root <none> and password file <none>
I20260807 08:29:57.316949 11112 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.317015 11112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:57.317214 11112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:57.319175 11112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "4d5dac72e18c46f4b94b07a54b42ea7d"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "6ff77a05f7a6347da7a72aae3116821c"
server_key_iv: "d6942aca32d9d18372db5121808808bd"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.319705 11112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "4d5dac72e18c46f4b94b07a54b42ea7d"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "6ff77a05f7a6347da7a72aae3116821c"
server_key_iv: "d6942aca32d9d18372db5121808808bd"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.323342 11112 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:29:57.325632 11139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.326845 11112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:57.326937 11112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "4d5dac72e18c46f4b94b07a54b42ea7d"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "6ff77a05f7a6347da7a72aae3116821c"
server_key_iv: "d6942aca32d9d18372db5121808808bd"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.327016 11112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.349481 11112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:57.352360 11112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:57.352504 11112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:57.352996 11112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:57.353741 11112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:57.353785 11112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.353822 11112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:57.353842 11112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.361901 11112 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39143
I20260807 08:29:57.361930 11307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39143 every 8 connection(s)
I20260807 08:29:57.362733 11112 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:57.368146 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 11112
I20260807 08:29:57.368269 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:29:57.368557 1653 external_mini_cluster.cc:1428] Setting key 45dd502fdd8c1e578d8d00841b3ca836
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:29:57.375133 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.364338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:39239 (local address 127.1.157.126:38219)
0807 08:29:57.364450 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.364454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.365423 (+ 969us) server_negotiation.cc:407] Connection header received
0807 08:29:57.366197 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.366199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.366246 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.366315 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.367355 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.367868 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.369311 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.369580 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.369673 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.371960 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.371963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.371965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.371988 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.373301 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.373787 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.373790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.373791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.373876 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.374255 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.374258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.374259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.374453 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.374529 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.374925 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.375016 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:29:57.375859 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.364649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:39239)
0807 08:29:57.365278 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.365317 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.366023 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.366469 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.366475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.366796 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.367231 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.367253 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.367981 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.367985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.369196 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.369206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.369716 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.369719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.369900 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.370425 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.370443 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.371821 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.373426 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.373430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.373441 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.373683 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.373988 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.373991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.373992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.374156 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.374535 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.374540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.374749 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.374895 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.375128 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":108,"threads_started":1}
I20260807 08:29:57.376828 11308 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
I20260807 08:29:57.377036 11308 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:57.377470 11308 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:57.378430 10639 ts_manager.cc:194] Registered new tserver with Master: 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:29:57.378930 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:39239
I20260807 08:29:57.381970 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.157.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.125.keytab.
Entry for principal oryx/127.1.157.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.125.keytab.
I20260807 08:29:57.406284 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:44145
--webserver_interface=127.1.157.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.125.keytab
--principal=oryx/127.1.157.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:38219,127.1.157.125:44145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:57.512595 11319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:57.512825 11319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:57.512866 11319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:57.517043 11319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:57.517107 11319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:57.517127 11319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:57.517145 11319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:57.517174 11319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:57.522353 11319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:38219,127.1.157.125:44145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.125.keytab
--principal=oryx/127.1.157.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:44145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.157.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
W20260807 08:29:57.522529 11319 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.157.126:38219,127.1.157.125:44145'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 08:29:57.523308 11319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:57.524238 11319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:57.532184 11333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.532163 11330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.532150 11331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.532411 11319 server_base.cc:1034] running on GCE node
I20260807 08:29:57.532964 11319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:57.533927 11319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:57.535077 11319 hybrid_clock.cc:648] HybridClock initialized: now 1786091397535058 us; error 31 us; skew 500 ppm
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:57.537626 11319 init.cc:377] Logged in from keytab as oryx/127.1.157.125@KRBTEST.COM (short username oryx)
I20260807 08:29:57.538439 11319 webserver.cc:460] Webserver started at http://127.1.157.125:41151/ using document root <none> and password file <none>
I20260807 08:29:57.538928 11319 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.538990 11319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:57.539167 11319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:57.541044 11319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/instance:
uuid: "98545d0754614e78a216b318544c8fc8"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "34bb44b3c999b001101eaa69fe040b0c"
server_key_iv: "beaf1e82755b2a4f15711f5d096c638f"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.541581 11319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal/instance:
uuid: "98545d0754614e78a216b318544c8fc8"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "34bb44b3c999b001101eaa69fe040b0c"
server_key_iv: "beaf1e82755b2a4f15711f5d096c638f"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.545009 11319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260807 08:29:57.547181 11345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.548257 11319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 08:29:57.548367 11319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "98545d0754614e78a216b318544c8fc8"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "34bb44b3c999b001101eaa69fe040b0c"
server_key_iv: "beaf1e82755b2a4f15711f5d096c638f"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.548453 11319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.556799 11319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:57.559602 11319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:57.559757 11319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:57.565887 11319 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.125:44145
I20260807 08:29:57.566155 11425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.125:44145 every 8 connection(s)
I20260807 08:29:57.566742 11319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
I20260807 08:29:57.569243 11427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:57.571565 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 11319
I20260807 08:29:57.572943 11427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.125@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
I20260807 08:29:57.583369 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.574509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.1.157.126:38219)
0807 08:29:57.574619 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.574622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.575014 (+ 392us) server_negotiation.cc:407] Connection header received
0807 08:29:57.575239 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.575242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.575288 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.575347 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.576093 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.576605 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.577570 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.577785 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.577872 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.580030 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.580034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.580037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.580067 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.581443 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.581881 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.581883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.581884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.581959 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.582309 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.582311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.582312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.582540 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.582622 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.583099 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.583218 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:29:57.584173 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.572957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.1.157.125:44145)
0807 08:29:57.573327 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.573342 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.573376 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:29:57.573976 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.573992 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.574445 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.574766 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.575618 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.576534 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.577337 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.577699 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.577846 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.580527 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.580534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.580546 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.580567 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.582556 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.582939 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.582944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.582946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.583026 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.583291 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.583295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.583297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.583536 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.583716 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.583876 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.583997 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":107,"threads_started":1}
I20260807 08:29:57.584184 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.574453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60164)
0807 08:29:57.574874 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.574924 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.575112 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.575427 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.575432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.575802 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.575954 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.575964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.576726 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.576729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.577452 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.577462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.577913 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.577917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.578016 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.578506 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.578520 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.579862 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.581552 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.581560 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.581567 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.581798 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.582053 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.582056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.582058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.582227 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.582638 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.582644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.582861 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.583036 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.583302 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":92,"threads_started":1}
I20260807 08:29:57.585533 11427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.125@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
I20260807 08:29:57.592579 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.585956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58748)
0807 08:29:57.586035 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.586047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.586114 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.586308 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.586310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.586458 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.586542 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.586547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.587183 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.587185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.587661 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.587668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.588492 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.588495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.588581 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.588981 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.588993 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.590048 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.591419 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.591421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.591423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.591605 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.591798 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.591800 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.591801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.591862 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.592166 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.592170 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.592251 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.592341 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.592451 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 08:29:57.592613 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.586018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.1.157.125:44145)
0807 08:29:57.586115 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.586119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.586131 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:57.586180 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.586182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.586220 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.586277 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.586652 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.587110 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.587736 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.588375 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.588453 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.590191 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.590193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.590195 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.590216 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.591328 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.591673 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.591675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.591676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.591739 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.591918 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.591920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.591921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.592061 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.592129 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.592382 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.592502 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":41}
I20260807 08:29:57.595220 11427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Bootstrap starting.
I20260807 08:29:57.597301 11427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:57.597965 11427 log.cc:824] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:57.599697 11427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: No bootstrap required, opened a new log
I20260807 08:29:57.602509 11427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } }
I20260807 08:29:57.602687 11427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:57.602741 11427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98545d0754614e78a216b318544c8fc8, State: Initialized, Role: FOLLOWER
I20260807 08:29:57.603107 11427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } }
I20260807 08:29:57.603538 11434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } } }
I20260807 08:29:57.603662 11434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:29:57.604238 11427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 08:29:57.608695 11447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:29:57.608776 11447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:29:57.610646 11319 master_runner.cc:428] Detected that this master 98545d0754614e78a216b318544c8fc8 is joining an existing cluster
I20260807 08:29:57.610692 11319 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.157.125:44145
I20260807 08:29:57.619029 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.612528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60170 (local address 127.1.157.126:38219)
0807 08:29:57.612690 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.612694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.612977 (+ 283us) server_negotiation.cc:407] Connection header received
0807 08:29:57.613083 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.613086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.613130 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.613193 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.613625 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.614146 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.614999 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.615227 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.615313 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.616278 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.616282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.616284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.616309 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.617546 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.617919 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.617923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.617925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.618000 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.618242 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.618245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.618247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.618464 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.618549 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.618796 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.618897 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 08:29:57.619035 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.612483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60170)
0807 08:29:57.612885 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.612896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.612988 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.613218 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.613220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.613410 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.613512 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.613517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.614247 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.614250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.614883 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.614892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.615336 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.615339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.615422 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.615840 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.615851 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.616150 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.617648 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.617651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.617653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.617835 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.618081 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.618084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.618086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.618163 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.618550 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.618553 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.618647 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.618732 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.618859 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":100,"threads_started":1}
I20260807 08:29:57.626237 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.612519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58752)
0807 08:29:57.619178 (+ 6659us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.619191 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.619294 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.619624 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.619626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.619785 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.619879 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.619886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.620818 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.620821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.621517 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.621529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.622089 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.622093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.622243 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.622664 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.622677 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.623013 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.624708 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.624712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.624713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.624959 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.625237 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.625240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.625242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.625327 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.625758 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.625761 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.625848 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.625943 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.626067 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6607,"mutex_wait_us":32,"thread_start_us":107,"threads_started":1}
I20260807 08:29:57.626279 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.612830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.1.157.125:44145)
0807 08:29:57.612931 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.612936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.619241 (+ 6305us) server_negotiation.cc:407] Connection header received
0807 08:29:57.619423 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.619426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.619464 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.619507 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.619985 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.620702 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.621646 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.621988 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.622087 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.623133 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.623136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.623138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.623160 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.624603 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.625062 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.625065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.625067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.625153 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.625452 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.625455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.625457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.625670 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.625758 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.625999 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.626135 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":40}
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.126@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
I20260807 08:29:57.633148 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.623299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58762)
0807 08:29:57.624266 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.624313 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.624441 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.624858 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.624863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.625320 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.625480 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.625488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.626432 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.626435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.627192 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.627203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.627753 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.627756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.627832 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.628287 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.628299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.629811 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.631713 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.631722 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.631726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.631940 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.632213 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.632215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.632217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.632307 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.632684 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.632688 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.632793 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.632900 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.633034 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":320,"threads_started":1}
I20260807 08:29:57.633217 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.623319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.1.157.125:44145)
0807 08:29:57.623611 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.623620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.624381 (+ 761us) server_negotiation.cc:407] Connection header received
0807 08:29:57.624567 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.624570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.624632 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.624707 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.625606 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.626322 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.627295 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.627649 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.627746 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.629933 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.629937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.629940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.629964 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.631595 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.632024 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.632027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.632029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.632106 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.632396 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.632399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.632401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.632597 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.632687 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.632960 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.633063 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
I20260807 08:29:57.634387 10639 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.157.125:44145: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "98545d0754614e78a216b318544c8fc8"
member_type: NON_VOTER
last_known_addr {
host: "127.1.157.125"
port: 44145
}
attrs {
promote: true
}
}
dest_uuid: "38352ac8d21c448f974575b7dc9f7d54"
cas_config_opid_index: -1
I20260807 08:29:57.635020 10639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } }
I20260807 08:29:57.636562 11464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 98545d0754614e78a216b318544c8fc8 (127.1.157.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } }
I20260807 08:29:57.637086 11385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:57.637928 11464 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.157.125:44145
I20260807 08:29:57.638396 11319 master.cc:537] Master@127.1.157.125:44145 shutting down...
I20260807 08:29:57.638999 11385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 08:29:57.639899 11464 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:29:57.641916 10617 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 08:29:57.642815 10617 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 -> Peer 98545d0754614e78a216b318544c8fc8 (127.1.157.125:44145): Couldn't send request to peer 98545d0754614e78a216b318544c8fc8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:29:57.643282 11465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } }
I20260807 08:29:57.643466 11465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:57.645239 11319 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:57.645500 11319 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:57.645603 11319 tablet_replica.cc:330] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: stopping tablet replica
WARNING: no policy specified for oryx/127.1.157.124@KRBTEST.COM; defaulting to no policy
I20260807 08:29:57.651455 11319 master.cc:552] Master@127.1.157.125:44145 shutdown complete.
I20260807 08:29:57.651495 11319 master_runner.cc:305] Clearing existing system tablet
I20260807 08:29:57.651667 11319 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.651705 11319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.124@KRBTEST.COM" created.
I20260807 08:29:57.653537 11319 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:57.654399 11470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.654630 11319 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.654759 11319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:57.654793 11319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "98545d0754614e78a216b318544c8fc8"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "34bb44b3c999b001101eaa69fe040b0c"
server_key_iv: "beaf1e82755b2a4f15711f5d096c638f"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.656757 11319 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:57.659035 11319 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:29:57.659121 11319 log.cc:1196] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 08:29:57.659394 11319 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Deleting consensus metadata
I20260807 08:29:57.659546 11319 master_runner.cc:315] Copying system tablet from 127.1.157.126:38219
I20260807 08:29:57.660750 11319 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: tablet copy: Beginning tablet copy session from remote peer at address 127.1.157.126:38219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.124.keytab.
Entry for principal oryx/127.1.157.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.124.keytab.
I20260807 08:29:57.667173 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:41513
--webserver_interface=127.1.157.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.124.keytab
--principal=oryx/127.1.157.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:29:57.668812 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.661282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60176)
0807 08:29:57.661614 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.661628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.661741 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.661959 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.661962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.662154 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.662251 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.662258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.663073 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.663077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.663977 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.663988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.664472 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.664476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.664564 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.665005 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.665019 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.665395 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.667256 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.667259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.667261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.667497 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.667784 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.667787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.667789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.667874 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.668262 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.668265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.668359 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.668497 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.668627 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260807 08:29:57.668843 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.661362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.1.157.126:38219)
0807 08:29:57.661484 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.661489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.661725 (+ 236us) server_negotiation.cc:407] Connection header received
0807 08:29:57.661825 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.661828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.661868 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.661932 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.662421 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.662971 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.664086 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.664371 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.664452 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.665565 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.665568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.665570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.665592 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.667139 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.667601 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.667604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.667606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.667694 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.667960 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.667963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.667965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.668170 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.668269 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.668570 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.668684 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:29:57.669553 10672 tablet_copy_service.cc:140] P 38352ac8d21c448f974575b7dc9f7d54: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 98545d0754614e78a216b318544c8fc8 ({username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:60176)
I20260807 08:29:57.669718 10672 tablet_copy_service.cc:161] P 38352ac8d21c448f974575b7dc9f7d54: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 98545d0754614e78a216b318544c8fc8 at {username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:60176: session id = 98545d0754614e78a216b318544c8fc8-00000000000000000000000000000000
I20260807 08:29:57.671669 10672 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:29:57.673627 11319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:29:57.675874 11319 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: tablet copy: Starting download of 0 data blocks...
I20260807 08:29:57.676056 11319 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: tablet copy: Starting download of 1 WAL segments...
I20260807 08:29:57.677889 11319 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:29:57.679596 10672 tablet_copy_service.cc:342] P 38352ac8d21c448f974575b7dc9f7d54: Request end of tablet copy session 98545d0754614e78a216b318544c8fc8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:60176
I20260807 08:29:57.679706 10672 tablet_copy_service.cc:433] P 38352ac8d21c448f974575b7dc9f7d54: ending tablet copy session 98545d0754614e78a216b318544c8fc8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 98545d0754614e78a216b318544c8fc8
I20260807 08:29:57.687871 11319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:57.689318 11485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.689318 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.689612 11486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.690035 11319 server_base.cc:1034] running on GCE node
I20260807 08:29:57.690181 11319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:57.690475 11319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:57.691605 11319 hybrid_clock.cc:648] HybridClock initialized: now 1786091397691593 us; error 26 us; skew 500 ppm
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:57.693077 11319 init.cc:377] Logged in from keytab as oryx/127.1.157.125@KRBTEST.COM (short username oryx)
I20260807 08:29:57.693495 11319 webserver.cc:460] Webserver started at http://127.1.157.125:39433/ using document root <none> and password file <none>
I20260807 08:29:57.693658 11319 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.693699 11319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:57.694867 11319 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:57.695760 11498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.696014 11319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:57.696086 11319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "98545d0754614e78a216b318544c8fc8"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "34bb44b3c999b001101eaa69fe040b0c"
server_key_iv: "beaf1e82755b2a4f15711f5d096c638f"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.696157 11319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.722563 11319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:57.724732 11319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:57.730304 11319 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.125:44145
I20260807 08:29:57.730357 11575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.125:44145 every 8 connection(s)
I20260807 08:29:57.730865 11319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
I20260807 08:29:57.732815 11576 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:29:57.733829 11576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Bootstrap starting.
I20260807 08:29:57.744571 11576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:29:57.744866 11576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Bootstrap complete.
I20260807 08:29:57.745398 11576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } }
I20260807 08:29:57.745532 11576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98545d0754614e78a216b318544c8fc8, State: Initialized, Role: LEARNER
I20260807 08:29:57.745627 11576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } }
I20260807 08:29:57.745958 11581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } }
I20260807 08:29:57.746057 11576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:57.746171 11581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:29:57.748934 11603 catalog_manager.cc:1259] Loaded cluster ID: fc29cca629f44c0cb4b812553eca344c
I20260807 08:29:57.749055 11603 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: loading cluster ID for follower catalog manager: success
I20260807 08:29:57.751269 11603 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: acquiring CA information for follower catalog manager: success
I20260807 08:29:57.752651 11603 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 08:29:57.807817 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:57.808044 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:57.808097 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:57.811921 11481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:29:57.811966 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:57.811993 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:57.812016 11481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:29:57.812050 11481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:29:57.816958 11481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.124.keytab
--principal=oryx/127.1.157.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:41513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.157.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:57.817824 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:57.818825 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:57.826562 11610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.826575 11609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:57.826567 11614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:57.826597 11481 server_base.cc:1034] running on GCE node
I20260807 08:29:57.827176 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:57.828153 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:57.829319 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786091397829303 us; error 27 us; skew 500 ppm
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:57.831818 11481 init.cc:377] Logged in from keytab as oryx/127.1.157.124@KRBTEST.COM (short username oryx)
I20260807 08:29:57.832587 11481 webserver.cc:460] Webserver started at http://127.1.157.124:40281/ using document root <none> and password file <none>
I20260807 08:29:57.833065 11481 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:57.833139 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:57.833349 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:29:57.835304 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/instance:
uuid: "f472cd9446d245b98ade5546adc77d96"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "e6ae445d35a38c017fcdf9fbaead1e8d"
server_key_iv: "5e8f854d55499a77d4db7dfcde1ffcd1"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.835863 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal/instance:
uuid: "f472cd9446d245b98ade5546adc77d96"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "e6ae445d35a38c017fcdf9fbaead1e8d"
server_key_iv: "5e8f854d55499a77d4db7dfcde1ffcd1"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.839389 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 08:29:57.841508 11627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:57.842639 11481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:29:57.842747 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "f472cd9446d245b98ade5546adc77d96"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "e6ae445d35a38c017fcdf9fbaead1e8d"
server_key_iv: "5e8f854d55499a77d4db7dfcde1ffcd1"
server_key_version: "encryptionkey@0"
I20260807 08:29:57.842831 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:57.868800 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:57.871219 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:57.871381 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:57.878432 11481 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.124:41513
I20260807 08:29:57.878669 11701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.124:41513 every 8 connection(s)
I20260807 08:29:57.879329 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
I20260807 08:29:57.882143 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:57.883920 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 11481
I20260807 08:29:57.885789 11703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.124@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:29:57.896522 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.888035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.1.157.126:38219)
0807 08:29:57.888153 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.888158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.888410 (+ 252us) server_negotiation.cc:407] Connection header received
0807 08:29:57.888527 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.888530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.888576 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.888643 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.889286 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.889782 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.890758 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.891076 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.891157 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.893600 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.893602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.893604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.893629 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.894941 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.895349 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.895352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.895353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.895427 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.895666 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.895668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.895669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.895818 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.895899 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.896236 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.896365 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:29:57.897197 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.885187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.1.157.124:41513)
0807 08:29:57.885563 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.885581 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.885616 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:29:57.886236 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.886252 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.886604 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.886877 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.887727 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.888971 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.889844 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.890401 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.890514 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.893066 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.893073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.893085 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.893111 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.894527 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.894906 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.894912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.894917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.895032 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.895292 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.895296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.895297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.895719 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.895904 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.896144 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.896418 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260807 08:29:57.897195 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.887945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60184)
0807 08:29:57.888301 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.888338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.888442 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.888698 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.888703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.889007 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.889154 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.889162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.889903 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.889906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.890643 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.890652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.891219 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.891223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.891345 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.891773 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.891794 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.893455 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.895032 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.895039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.895044 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.895260 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.895505 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.895508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.895509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.895592 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.895919 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.895925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.896047 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.896167 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.896489 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":40,"thread_start_us":102,"threads_started":1}
I20260807 08:29:57.898362 11703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.124@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
I20260807 08:29:57.905776 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.898751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58766)
0807 08:29:57.898844 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.898856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.898917 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.899368 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.899371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.899510 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.899588 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.899593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.900447 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.900450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.900951 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.900959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.901388 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.901391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.901479 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.901839 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.901852 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.902944 (+ 1092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.904511 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.904514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.904516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.904690 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.904940 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.904943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.904945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.905021 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.905350 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.905354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.905436 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.905517 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.905642 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:29:57.905851 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.898871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.1.157.125:44145)
0807 08:29:57.899133 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.899137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.899150 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:29:57.899191 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.899194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.899240 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.899330 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.899698 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.900349 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.901055 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.901295 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.901381 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.903068 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.903070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.903072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.903094 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.904410 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.904782 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.904784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.904785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.904859 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.905098 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.905101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.905103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.905263 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.905344 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.905584 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.905698 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":102,"threads_started":1}
I20260807 08:29:57.906476 11703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091397, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.124@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
I20260807 08:29:57.913267 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.906857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37138)
0807 08:29:57.906930 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.906942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.907008 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.907186 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.907189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.907322 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.907378 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.907382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.908492 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.908495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.908957 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.908965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.909372 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.909375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.909469 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.909775 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.909785 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.910895 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.912087 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.912090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.912092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.912257 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.912476 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.912478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.912480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.912546 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.912846 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.912849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.912924 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.913003 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.913118 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":16}
I20260807 08:29:57.913295 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.906914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.1.157.124:41513)
0807 08:29:57.907008 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.907012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.907024 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:57.907073 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.907076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.907107 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.907168 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.907471 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.907999 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.909061 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.909284 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.909363 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.911005 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.911008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.911009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.911028 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.911996 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.912335 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.912338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.912339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.912401 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.912619 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.912622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.912624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.912764 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.912842 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.913055 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.913168 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":40}
I20260807 08:29:57.915707 11703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Bootstrap starting.
I20260807 08:29:57.917726 11703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:57.918418 11703 log.cc:824] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:57.920046 11703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: No bootstrap required, opened a new log
I20260807 08:29:57.922857 11703 raft_consensus.cc:348] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } }
I20260807 08:29:57.923023 11703 raft_consensus.cc:374] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:57.923081 11703 raft_consensus.cc:729] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f472cd9446d245b98ade5546adc77d96, State: Initialized, Role: FOLLOWER
I20260807 08:29:57.923445 11703 consensus_queue.cc:260] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } }
I20260807 08:29:57.923838 11711 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } } }
I20260807 08:29:57.923945 11711 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:29:57.924463 11703 sys_catalog.cc:564] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 08:29:57.929062 11732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:29:57.929133 11732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:29:57.931908 11481 master_runner.cc:428] Detected that this master f472cd9446d245b98ade5546adc77d96 is joining an existing cluster
I20260807 08:29:57.931959 11481 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.157.124:41513
I20260807 08:29:57.941497 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.933846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37150)
0807 08:29:57.934554 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.934566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.934657 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.934900 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.934902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.935112 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.935216 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.935223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.936223 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.936226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.937000 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.937008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.937465 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.937468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.938002 (+ 534us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.938392 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.938405 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.938706 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.940316 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.940318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.940320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.940494 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.940732 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.940734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.940735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.940802 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.941144 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.941146 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.941214 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.941293 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.941380 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":26,"thread_start_us":116,"threads_started":1}
I20260807 08:29:57.941612 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.933936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.1.157.124:41513)
0807 08:29:57.934024 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.934027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.934617 (+ 590us) server_negotiation.cc:407] Connection header received
0807 08:29:57.934731 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.934734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.934764 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.934802 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.935334 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.936102 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.937088 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.937298 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.937375 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.938831 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.938835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.938838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.938863 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.940221 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.940573 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.940576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.940578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.940657 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.940871 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.940874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.940876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.941055 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.941159 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.941354 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.941453 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:29:57.950192 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 10727
I20260807 08:29:57.973429 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.934195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.1.157.125:44145)
0807 08:29:57.934294 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.934298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.934310 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:57.934370 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.934373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.934416 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.934503 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.935234 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.935823 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.936921 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.937274 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.937377 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.970016 (+ 32639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.970020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.970023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.970049 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.971663 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.972143 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.972146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.972148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.972229 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.972529 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.972532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.972534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.972736 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.972813 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.973149 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.973275 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:29:57.973961 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.933839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58772)
0807 08:29:57.934122 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.934136 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.934219 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.934642 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.934645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.934968 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.935107 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.935115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.935921 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.935924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.936798 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.936808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.937395 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.937398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.937477 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.937987 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.938001 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.969839 (+ 31838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.971795 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.971798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.971800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.972035 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.972335 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.972338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.972339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.972435 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.972884 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.972887 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.972976 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.973612 (+ 636us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.973782 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":97,"threads_started":1}
I20260807 08:29:57.975596 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:46013
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:38219
--webserver_port=40841
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:29:57.978873 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.933847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60194)
0807 08:29:57.934334 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.934365 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.934541 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.934887 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.934890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.968953 (+ 34063us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.969080 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.969088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.970141 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.970144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.971507 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.971519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.972181 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.972184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.972266 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.973242 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.973255 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.973893 (+ 638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.977513 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.977515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.977517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.977697 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.977958 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.977961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.977963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.978042 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.978408 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.978411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.978501 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.978598 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.978717 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":80,"threads_started":1}
I20260807 08:29:57.978976 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.933815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.1.157.126:38219)
0807 08:29:57.933933 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.933937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.934429 (+ 492us) server_negotiation.cc:407] Connection header received
0807 08:29:57.934654 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.934658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.934702 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.934747 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.969241 (+ 34494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.969997 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.971644 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.971966 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.972035 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.974038 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.974042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.974045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.974071 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.977405 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.977801 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.977804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.977805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.977879 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.978129 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.978132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.978134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.978300 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.978405 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.978697 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.978825 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 08:29:57 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091396, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.126@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
I20260807 08:29:57.991528 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.983328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.1.157.124:41513)
0807 08:29:57.983434 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.983438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.983452 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:29:57.983518 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.983521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.983571 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.983655 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:57.984068 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.984771 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.985838 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.986149 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.986248 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.988421 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:57.988425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:57.988428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:57.988453 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:57.989909 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.990406 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.990410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.990411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.990492 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:57.990760 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:57.990763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:57.990765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:57.990950 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:57.991041 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:57.991293 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 08:29:57.991396 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:29:57.991715 11711 raft_consensus.cc:482] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:57.991863 11711 raft_consensus.cc:504] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } }
I20260807 08:29:57.992810 11711 leader_election.cc:288] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219), 98545d0754614e78a216b318544c8fc8 (127.1.157.125:44145)
I20260807 08:29:57.993310 11537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f472cd9446d245b98ade5546adc77d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98545d0754614e78a216b318544c8fc8" is_pre_election: true
I20260807 08:29:57.993311 10658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f472cd9446d245b98ade5546adc77d96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38352ac8d21c448f974575b7dc9f7d54" is_pre_election: true
I20260807 08:29:57.993481 11537 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Handling vote request from an unknown peer f472cd9446d245b98ade5546adc77d96
I20260807 08:29:57.993599 11537 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate f472cd9446d245b98ade5546adc77d96 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 08:29:57.991506 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.983133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37162)
0807 08:29:57.983244 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.983259 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.983330 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:57.983674 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:57.983677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:57.983863 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:57.983957 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.983964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.984876 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.984879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.985711 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:57.985721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:57.986263 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:57.986267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:57.986380 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:57.986802 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:57.986815 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:57.988266 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:57.990027 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.990030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.990032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.990284 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.990578 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:57.990581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:57.990583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:57.990674 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:57.991036 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:57.991039 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:57.991128 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 08:29:57.991222 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:29:57.991354 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:29:57.994228 11630 leader_election.cc:302] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f472cd9446d245b98ade5546adc77d96; no voters: 38352ac8d21c448f974575b7dc9f7d54, 98545d0754614e78a216b318544c8fc8
I20260807 08:29:57.995035 10639 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.157.124:41513: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f472cd9446d245b98ade5546adc77d96"
member_type: NON_VOTER
last_known_addr {
host: "127.1.157.124"
port: 41513
}
attrs {
promote: true
}
}
dest_uuid: "38352ac8d21c448f974575b7dc9f7d54"
cas_config_opid_index: 5
I20260807 08:29:57.995577 10639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } }
I20260807 08:29:57.996555 11464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f472cd9446d245b98ade5546adc77d96 (127.1.157.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:57.997430 11666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:57.997982 11464 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.157.124:41513
I20260807 08:29:57.998134 11464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:57.998219 11464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:57.998821 11666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:29:57.999183 11481 master.cc:537] Master@127.1.157.124:41513 shutting down...
I20260807 08:29:57.999338 11465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:58.004144 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.998488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.1.157.125:44145)
0807 08:29:57.998583 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:57.998587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:57.999696 (+ 1109us) server_negotiation.cc:407] Connection header received
0807 08:29:57.999871 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:57.999874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:57.999925 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:57.999983 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.000396 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.001169 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.002761 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.003385 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.003461 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.003597 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.003959 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.004032 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 08:29:58.004161 10620 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 -> Peer f472cd9446d245b98ade5546adc77d96 (127.1.157.124:41513): Couldn't send request to peer f472cd9446d245b98ade5546adc77d96. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 08:29:58.004292 11711 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 08:29:58.004695 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:57.996809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58788)
0807 08:29:57.999620 (+ 2811us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:57.999633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:57.999768 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.000115 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.000118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.000134 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.000211 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.000216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.001310 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.001313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.002610 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.002622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.003693 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.003697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.003803 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.003848 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.004488 (+ 640us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.004567 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2746,"thread_start_us":88,"threads_started":1}
I20260807 08:29:58.005250 11537 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 08:29:58.005720 11465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 08:29:58.006204 11481 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:58.006489 11481 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:58.006594 11481 tablet_replica.cc:330] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: stopping tablet replica
I20260807 08:29:58.007510 11537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f472cd9446d245b98ade5546adc77d96 (127.1.157.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.009238 11581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38352ac8d21c448f974575b7dc9f7d54. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.009341 11581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:29:58.009415 11581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: NON_VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: true } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.009475 11581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:29:58.015440 11465 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: attempting to promote NON_VOTER 98545d0754614e78a216b318544c8fc8 to VOTER
I20260807 08:29:58.015865 11465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } }
I20260807 08:29:58.017225 11537 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 LEARNER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 08:29:58.017470 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.016798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37176)
0807 08:29:58.017018 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.017398 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 08:29:58.017689 11754 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 08:29:58.018124 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.017898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37186)
0807 08:29:58.018021 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.018065 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 08:29:58.018231 10620 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 -> Peer f472cd9446d245b98ade5546adc77d96 (127.1.157.124:41513): Couldn't send request to peer f472cd9446d245b98ade5546adc77d96. Status: Network error: Client connection negotiation failed: client connection to 127.1.157.124:41513: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:29:58.019372 11464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 98545d0754614e78a216b318544c8fc8 (127.1.157.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.020802 11464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.020910 11464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.021039 11464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.021132 11464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.021898 11537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 98545d0754614e78a216b318544c8fc8 (127.1.157.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.022264 11481 master.cc:552] Master@127.1.157.124:41513 shutdown complete.
I20260807 08:29:58.022302 11481 master_runner.cc:305] Clearing existing system tablet
I20260807 08:29:58.022636 11481 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:58.022689 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:58.023595 11753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.023721 11753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:29:58.026643 11481 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260807 08:29:58.027484 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.027696 11481 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:58.027822 11481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:58.027855 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "f472cd9446d245b98ade5546adc77d96"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "e6ae445d35a38c017fcdf9fbaead1e8d"
server_key_iv: "5e8f854d55499a77d4db7dfcde1ffcd1"
server_key_version: "encryptionkey@0"
I20260807 08:29:58.029675 11481 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:58.032405 11481 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 08:29:58.032557 11481 log.cc:1196] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 08:29:58.032856 11481 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Deleting consensus metadata
I20260807 08:29:58.033057 11481 master_runner.cc:315] Copying system tablet from 127.1.157.126:38219
I20260807 08:29:58.034168 11481 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: tablet copy: Beginning tablet copy session from remote peer at address 127.1.157.126:38219
I20260807 08:29:58.051129 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.042733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60204 (local address 127.1.157.126:38219)
0807 08:29:58.042864 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.042868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.043039 (+ 171us) server_negotiation.cc:407] Connection header received
0807 08:29:58.043161 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.043164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.043214 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.043306 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.043774 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.044584 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.045731 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.046083 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.046186 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.047633 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.047637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.047640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.047668 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.049281 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.049760 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.049763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.049765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.049858 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.050153 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.050157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.050158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.050393 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.050498 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.050788 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.050975 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:29:58.051149 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.042642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60204)
0807 08:29:58.042945 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.042959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.043070 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.043333 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.043336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.043544 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.043652 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.043660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.044702 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.044706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.045606 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.045617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.046218 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.046223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.046316 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.047015 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.047030 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.047480 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.049401 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.049405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.049407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.049656 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.049954 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.049957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.049959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.050057 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.050489 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.050492 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.050607 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.050712 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.050849 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260807 08:29:58.051859 10672 tablet_copy_service.cc:140] P 38352ac8d21c448f974575b7dc9f7d54: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f472cd9446d245b98ade5546adc77d96 ({username='oryx', principal='oryx/127.1.157.124@KRBTEST.COM'} at 127.0.0.1:60204)
I20260807 08:29:58.051952 10672 tablet_copy_service.cc:161] P 38352ac8d21c448f974575b7dc9f7d54: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f472cd9446d245b98ade5546adc77d96 at {username='oryx', principal='oryx/127.1.157.124@KRBTEST.COM'} at 127.0.0.1:60204: session id = f472cd9446d245b98ade5546adc77d96-00000000000000000000000000000000
I20260807 08:29:58.053033 10672 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 08:29:58.054301 11481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 08:29:58.056589 11481 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: tablet copy: Starting download of 0 data blocks...
I20260807 08:29:58.056782 11481 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: tablet copy: Starting download of 1 WAL segments...
I20260807 08:29:58.058219 11481 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 08:29:58.059973 10672 tablet_copy_service.cc:342] P 38352ac8d21c448f974575b7dc9f7d54: Request end of tablet copy session f472cd9446d245b98ade5546adc77d96-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.157.124@KRBTEST.COM'} at 127.0.0.1:60204
I20260807 08:29:58.060060 10672 tablet_copy_service.cc:433] P 38352ac8d21c448f974575b7dc9f7d54: ending tablet copy session f472cd9446d245b98ade5546adc77d96-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f472cd9446d245b98ade5546adc77d96
I20260807 08:29:58.068320 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:58.069937 11768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.069994 11769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.070111 11771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:58.070497 11481 server_base.cc:1034] running on GCE node
I20260807 08:29:58.070672 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:58.070986 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:58.072144 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786091398072130 us; error 29 us; skew 500 ppm
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:58.073724 11481 init.cc:377] Logged in from keytab as oryx/127.1.157.124@KRBTEST.COM (short username oryx)
I20260807 08:29:58.074159 11481 webserver.cc:460] Webserver started at http://127.1.157.124:44637/ using document root <none> and password file <none>
I20260807 08:29:58.074378 11481 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:58.074426 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:58.075682 11481 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:58.076653 11782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.076925 11481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 08:29:58.077003 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "f472cd9446d245b98ade5546adc77d96"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "e6ae445d35a38c017fcdf9fbaead1e8d"
server_key_iv: "5e8f854d55499a77d4db7dfcde1ffcd1"
server_key_version: "encryptionkey@0"
I20260807 08:29:58.077083 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:58.097615 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:58.099437 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:58.104135 11481 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.124:41513
I20260807 08:29:58.104872 11864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.124:41513 every 8 connection(s)
I20260807 08:29:58.104915 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
I20260807 08:29:58.106312 11866 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:29:58.107105 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Bootstrap starting.
W20260807 08:29:58.110208 11750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:58.110457 11750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:58.110512 11750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:58.114331 11750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:58.114404 11750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:58.114478 11750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:29:58.115345 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:29:58.115561 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: Bootstrap complete.
I20260807 08:29:58.115955 11866 raft_consensus.cc:348] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } }
I20260807 08:29:58.116063 11866 raft_consensus.cc:729] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f472cd9446d245b98ade5546adc77d96, State: Initialized, Role: LEARNER
I20260807 08:29:58.116155 11866 consensus_queue.cc:260] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } }
I20260807 08:29:58.116421 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.116535 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:29:58.116547 11866 sys_catalog.cc:564] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:29:58.118963 11888 catalog_manager.cc:1259] Loaded cluster ID: fc29cca629f44c0cb4b812553eca344c
I20260807 08:29:58.119009 11888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: loading cluster ID for follower catalog manager: success
I20260807 08:29:58.120033 11750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:46013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=40841
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:58.120764 11888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: acquiring CA information for follower catalog manager: success
I20260807 08:29:58.121004 11750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:58.121850 11888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:29:58.122040 11750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:58.130821 11891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.130837 11890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.130807 11894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:58.131155 11750 server_base.cc:1034] running on GCE node
I20260807 08:29:58.131522 11750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:58.132054 11750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:58.133209 11750 hybrid_clock.cc:648] HybridClock initialized: now 1786091398133187 us; error 33 us; skew 500 ppm
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:58.135761 11750 init.cc:377] Logged in from keytab as oryx/127.1.157.65@KRBTEST.COM (short username oryx)
I20260807 08:29:58.136530 11750 webserver.cc:460] Webserver started at http://127.1.157.65:40841/ using document root <none> and password file <none>
I20260807 08:29:58.136996 11750 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:58.137070 11750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:58.140700 11750 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:58.143003 11901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.144073 11750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:29:58.144163 11750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "24623ec868a04df390a3223e0c079963"
format_stamp: "Formatted at 2026-08-07 08:29:56 on dist-test-slave-6zbq"
server_key: "ddb241b0c828eb0c16d6f595311e2a1a"
server_key_iv: "139b36f3e6b865df2f3d134cfc0fe4b2"
server_key_version: "encryptionkey@0"
I20260807 08:29:58.144615 11750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:58.154970 11750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:58.157568 11750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:58.157716 11750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:58.158195 11750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:58.158963 11750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:58.159009 11750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.159056 11750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:58.159082 11750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.167300 11750 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:46013
I20260807 08:29:58.167351 12067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:46013 every 8 connection(s)
I20260807 08:29:58.168145 11750 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:29:58.172561 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 11750
I20260807 08:29:58.172744 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 10921
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
I20260807 08:29:58.191841 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.169705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:54431 (local address 127.1.157.124:41513)
0807 08:29:58.170084 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.170089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.171250 (+ 1161us) server_negotiation.cc:407] Connection header received
0807 08:29:58.171828 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.171831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.171888 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.171966 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.183189 (+ 11223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.184177 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.185529 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.185807 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.185870 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.188477 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.188481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.188483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.188506 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.189942 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.190453 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.190456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.190457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.190539 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.190847 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.190850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.190851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.191017 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.191076 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.191594 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.191697 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":78,"threads_started":1}
I20260807 08:29:58.193074 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.170136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.1.157.65:54431)
0807 08:29:58.171160 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.171171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.171652 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.172135 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.172141 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.172663 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.182985 (+ 10322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.183007 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.184312 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.184315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.185409 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.185419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.186040 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.186043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.186194 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.186789 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.186806 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.188318 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.190051 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.190056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.190068 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.190358 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.190625 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.190627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.190629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.190764 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.191127 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.191133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.191381 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.192012 (+ 631us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.192300 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":195,"spinlock_wait_cycles":19200,"thread_start_us":131,"threads_started":1}
I20260807 08:29:58.194123 12069 heartbeater.cc:347] Connected to a master server at 127.1.157.124:41513
I20260807 08:29:58.195127 12069 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.195230 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.170332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:48445 (local address 127.1.157.125:44145)
0807 08:29:58.170473 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.170477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.170843 (+ 366us) server_negotiation.cc:407] Connection header received
0807 08:29:58.172035 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.172039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.172095 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.172145 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.183126 (+ 10981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.183928 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.185838 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.186276 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.186406 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.190282 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.190287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.190290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.190323 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.192329 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.193557 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.193561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.193562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.193680 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.194060 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.194064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.194067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.194322 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.194503 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.194934 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.195070 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:29:58.195639 12069 heartbeater.cc:511] Master 127.1.157.124:41513 requested a full tablet report, sending...
I20260807 08:29:58.196172 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.170136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.1.157.65:48445)
0807 08:29:58.170708 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.170755 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.171923 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.172285 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.172287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.172713 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.182981 (+ 10268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.182998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.184139 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.184143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.185671 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.185684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.186931 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.186936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.187049 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.187576 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.187593 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.190083 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.192887 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.192891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.192893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.193417 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.193813 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.193817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.193819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.193930 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.194574 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.194577 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.194702 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.195745 (+ 1043us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.195951 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":48,"spinlock_wait_cycles":4864,"thread_start_us":163,"threads_started":1}
I20260807 08:29:58.196960 11809 ts_manager.cc:194] Registered new tserver with Master: 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013)
I20260807 08:29:58.197321 12071 heartbeater.cc:347] Connected to a master server at 127.1.157.125:44145
I20260807 08:29:58.197466 12071 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.197719 12071 heartbeater.cc:511] Master 127.1.157.125:44145 requested a full tablet report, sending...
I20260807 08:29:58.198972 11523 ts_manager.cc:194] Registered new tserver with Master: 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013)
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:29:58.206032 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:36719
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:38219
--webserver_port=34265
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:29:58.206393 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.169707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:47171 (local address 127.1.157.126:38219)
0807 08:29:58.169955 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.169959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.171047 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 08:29:58.171843 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.171847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.171904 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.171980 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.200137 (+ 28157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.200653 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.201406 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.201626 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.201712 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.203602 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.203606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.203609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.203635 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.204793 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.205185 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.205188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.205189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.205274 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.205523 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.205526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.205528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.205736 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.205837 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.206092 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.206217 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260807 08:29:58.206566 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.170136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:47171)
0807 08:29:58.170967 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.170978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.171655 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.172134 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.172143 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.172664 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.200008 (+ 27344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.200014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.200770 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.200773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.201291 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.201298 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.201742 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.201746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.201837 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.202241 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.202251 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.203464 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.204893 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.204896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.204899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.205086 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.205355 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.205358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.205360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.205435 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.205841 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.205844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.205918 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.205993 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.206093 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":28,"spinlock_wait_cycles":23680,"thread_start_us":177,"threads_started":1}
I20260807 08:29:58.207195 12070 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
I20260807 08:29:58.207281 12070 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.207433 12070 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:58.208221 10639 ts_manager.cc:194] Re-registered known tserver with Master: 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013)
I20260807 08:29:58.208966 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:47171
W20260807 08:29:58.312333 12080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:58.312541 12080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:58.312584 12080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:58.316485 12080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:58.316530 12080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:58.316596 12080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:29:58.321666 12080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:36719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=34265
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:58.322543 12080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:58.323416 12080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:58.331974 12086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.332046 12089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.331981 12091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:58.332237 12080 server_base.cc:1034] running on GCE node
I20260807 08:29:58.332583 12080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:58.333168 12080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:58.334321 12080 hybrid_clock.cc:648] HybridClock initialized: now 1786091398334308 us; error 26 us; skew 500 ppm
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:58.336823 12080 init.cc:377] Logged in from keytab as oryx/127.1.157.66@KRBTEST.COM (short username oryx)
I20260807 08:29:58.337559 12080 webserver.cc:460] Webserver started at http://127.1.157.66:34265/ using document root <none> and password file <none>
I20260807 08:29:58.338018 12080 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:58.338083 12080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:58.341477 12080 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:58.343283 12102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.344353 12080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:58.344445 12080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "8f63837b354c40a8afe022d9db8369b2"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "ce7816333a3562a62e5db2f6d6051c4e"
server_key_iv: "a7e596c33e401bc3c04e120a119a684d"
server_key_version: "encryptionkey@0"
I20260807 08:29:58.344908 12080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:58.380786 11308 heartbeater.cc:503] Master 127.1.157.126:38219 was elected leader, sending a full tablet report...
I20260807 08:29:58.381879 12080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:58.384944 12080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:58.385083 12080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:58.385553 12080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:58.386292 12080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:58.386354 12080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.386392 12080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:58.386412 12080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.394668 12080 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:36719
I20260807 08:29:58.394721 12268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:36719 every 8 connection(s)
I20260807 08:29:58.395524 12080 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:29:58.402635 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 12080
I20260807 08:29:58.402807 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 11112
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:29:58.410315 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:48305 (local address 127.1.157.125:44145)
0807 08:29:58.397917 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.397920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.398984 (+ 1064us) server_negotiation.cc:407] Connection header received
0807 08:29:58.399526 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.399529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.399578 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.399628 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.401851 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.402336 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.403496 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.403747 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.403805 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.406986 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.406989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.406991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.407015 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.408399 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.408896 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.408899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.408900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.408977 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.409282 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.409285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.409286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.409466 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.409518 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.410045 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.410164 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:29:58.411639 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57313 (local address 127.1.157.126:38219)
0807 08:29:58.397842 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.397846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.399198 (+ 1352us) server_negotiation.cc:407] Connection header received
0807 08:29:58.399526 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.399529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.399579 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.399628 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.400906 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.401715 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.403447 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.403893 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.403979 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.407248 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.407267 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.407269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.407302 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.409095 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.409696 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.409701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.409703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.409814 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.410189 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.410194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.410196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.410508 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.410609 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.411024 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.411171 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:29:58.412139 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.1.157.66:48305)
0807 08:29:58.398896 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.398906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.399329 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.399814 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.399821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.400274 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.400760 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.400766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.402508 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.402512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.403380 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.403391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.403962 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.403965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.404125 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.404621 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.404637 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.406825 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.408516 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.408521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.408533 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.408805 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.409064 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.409066 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.409068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.409200 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.409579 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.409584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.409818 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.410743 (+ 925us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.411044 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"mutex_wait_us":569,"spinlock_wait_cycles":28800,"thread_start_us":138,"threads_started":1}
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
I20260807 08:29:58.412631 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:57313)
0807 08:29:58.399110 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.399119 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.399332 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.399814 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.399821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.400264 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.400756 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.400766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.401856 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.401859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.403294 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.403307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.404149 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.404152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.404260 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.404999 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.405016 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.407070 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.409243 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.409247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.409250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.409557 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.409939 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.409942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.409944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.410056 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.410685 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.410688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.410791 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.411875 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.412037 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1534,"mutex_wait_us":708,"spinlock_wait_cycles":11008,"thread_start_us":190,"threads_started":1}
I20260807 08:29:58.413722 12278 heartbeater.cc:347] Connected to a master server at 127.1.157.125:44145
I20260807 08:29:58.413730 12273 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
I20260807 08:29:58.413985 12273 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.413986 12278 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.414513 12273 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:58.414536 12278 heartbeater.cc:511] Master 127.1.157.125:44145 requested a full tablet report, sending...
I20260807 08:29:58.415669 10639 ts_manager.cc:194] Re-registered known tserver with Master: 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:58.416098 11523 ts_manager.cc:194] Registered new tserver with Master: 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:58.416571 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:57313
I20260807 08:29:58.417510 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57563 (local address 127.1.157.124:41513)
0807 08:29:58.397993 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.397995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.398589 (+ 594us) server_negotiation.cc:407] Connection header received
0807 08:29:58.399516 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.399519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.399576 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.399649 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.400886 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.401714 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.410311 (+ 8597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.410679 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.410756 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.413826 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.413829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.413831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.413860 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.415496 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.415998 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.416002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.416004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.416086 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.416447 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.416450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.416452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.416699 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.416772 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.417267 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.417375 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 08:29:58.417519 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.397509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.1.157.66:57563)
0807 08:29:58.398452 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.398499 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.399327 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.399813 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.399819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.400243 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.400742 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.400756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.401856 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.401859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.410183 (+ 8324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.410193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.410965 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.410968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.411063 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.411522 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.411535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.413224 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.415591 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.415594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.415595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.415890 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.416248 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.416250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.416252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.416342 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.417005 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.417007 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.417095 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.417193 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.417356 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":92,"thread_start_us":599,"threads_started":1}
I20260807 08:29:58.429358 12271 heartbeater.cc:347] Connected to a master server at 127.1.157.124:41513
I20260807 08:29:58.429445 12271 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.429613 12271 heartbeater.cc:511] Master 127.1.157.124:41513 requested a full tablet report, sending...
I20260807 08:29:58.430224 11809 ts_manager.cc:194] Registered new tserver with Master: 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:58.434041 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:39143
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:38219
--webserver_port=40703
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:34461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:29:58.539060 12283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:29:58.539276 12283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:29:58.539316 12283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:29:58.543165 12283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:29:58.543210 12283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:29:58.543274 12283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:29:58.548403 12283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:34461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:39143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=40703
--tserver_master_addrs=127.1.157.126:38219,127.1.157.125:44145,127.1.157.124:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:29:58.549320 12283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:29:58.550223 12283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:29:58.558985 12292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:29:58.559057 12293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:58.559077 12283 server_base.cc:1034] running on GCE node
W20260807 08:29:58.559290 12295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:29:58.559656 12283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:29:58.560237 12283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:29:58.561426 12283 hybrid_clock.cc:648] HybridClock initialized: now 1786091398561410 us; error 28 us; skew 500 ppm
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:29:58.563987 12283 init.cc:377] Logged in from keytab as oryx/127.1.157.67@KRBTEST.COM (short username oryx)
I20260807 08:29:58.564750 12283 webserver.cc:460] Webserver started at http://127.1.157.67:40703/ using document root <none> and password file <none>
I20260807 08:29:58.565239 12283 fs_manager.cc:359] Metadata directory not provided
I20260807 08:29:58.565330 12283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:29:58.568895 12283 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:58.570911 12307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.572009 12283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:29:58.572114 12283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "4d5dac72e18c46f4b94b07a54b42ea7d"
format_stamp: "Formatted at 2026-08-07 08:29:57 on dist-test-slave-6zbq"
server_key: "6ff77a05f7a6347da7a72aae3116821c"
server_key_iv: "d6942aca32d9d18372db5121808808bd"
server_key_version: "encryptionkey@0"
I20260807 08:29:58.572609 12283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:29:58.598614 12283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:29:58.601146 12283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:29:58.601286 12283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:29:58.601750 12283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:29:58.602530 12283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:29:58.602576 12283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.602612 12283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:29:58.602631 12283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:29:58.612905 12283 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39143
I20260807 08:29:58.614130 12283 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:29:58.619912 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 12283
I20260807 08:29:58.620090 12477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39143 every 8 connection(s)
I20260807 08:29:58.632650 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.622228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.1.157.125:44145)
0807 08:29:58.622362 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.622366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.622936 (+ 570us) server_negotiation.cc:407] Connection header received
0807 08:29:58.623151 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.623155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.623208 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.623265 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.624335 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.625115 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.626252 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.626619 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.626693 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.628620 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.628624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.628626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.628651 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.630830 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.631316 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.631320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.631322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.631414 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.631721 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.631725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.631727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.631945 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.632042 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.632380 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.632493 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:29:58.633625 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.621912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.1.157.126:38219)
0807 08:29:58.622130 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.622134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.624536 (+ 2402us) server_negotiation.cc:407] Connection header received
0807 08:29:58.624672 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.624675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.624729 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.624778 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.625735 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.626480 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.627563 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.627905 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.627973 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.630202 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.630206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.630208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.630238 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.631833 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.632306 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.632310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.632312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.632399 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.632690 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.632694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.632696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.632909 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.633020 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.633374 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.633482 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.125@KRBTEST.COM
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
Aug 07 08:29:58 dist-test-slave-6zbq krb5kdc[10573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091398, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.124@KRBTEST.COM
I20260807 08:29:58.637604 10639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:29:58.641543 10639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:29:58.643925 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.635389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.1.157.124:41513)
0807 08:29:58.639839 (+ 4450us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.639843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.639855 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:29:58.639895 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.639898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.639955 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.640058 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.640298 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.640819 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.642788 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.643413 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.643515 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.643654 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.643743 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.643809 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4382,"thread_start_us":62,"threads_started":1}
I20260807 08:29:58.643939 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.635323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37208)
0807 08:29:58.639542 (+ 4219us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.639558 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.639651 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.640094 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.640097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.640115 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.640199 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.640203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.641327 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.641330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.642662 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.642673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.643533 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.643537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.643631 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.643657 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.643732 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.643806 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4150,"thread_start_us":61,"threads_started":1}
I20260807 08:29:58.643944 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.621876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.1.157.67:34319)
0807 08:29:58.623624 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.623677 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.624533 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.625031 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.625039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.625592 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.626313 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.626332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.627488 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.627491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.628808 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.628819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.631461 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.631465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.631663 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.634243 (+ 2580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.634257 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.638963 (+ 4706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.640930 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.640936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.640950 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.641216 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.641521 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.641524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.641526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.641708 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.642166 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.642173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.642475 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.642671 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.642992 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1527,"mutex_wait_us":46,"spinlock_wait_cycles":6016,"thread_start_us":677,"threads_started":1}
I20260807 08:29:58.644860 11826 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 08:29:58.645094 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.621388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:34319 (local address 127.1.157.124:41513)
0807 08:29:58.621525 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.621529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.623764 (+ 2235us) server_negotiation.cc:407] Connection header received
0807 08:29:58.624735 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.624738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.624782 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.624851 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.626462 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.627241 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.628919 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.629250 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.629383 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.639092 (+ 9709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.639096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.639098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.639126 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.640814 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.641327 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.641330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.641332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.641423 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.641822 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.641826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.641827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.642033 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.642114 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.644859 (+ 2745us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.644961 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:29:58.645790 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.622974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.1.157.124:41513)
0807 08:29:58.623238 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.623243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.628915 (+ 5672us) server_negotiation.cc:407] Connection header received
0807 08:29:58.629084 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.629087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.629176 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.629263 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.634648 (+ 5385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.635393 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.636480 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.636755 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.636843 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.638227 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.638231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.638233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.638258 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.641530 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.642993 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.642997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.642998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.643081 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.643994 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.643998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.644000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.644174 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.644258 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.644553 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.645644 (+ 1091us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":107,"threads_started":1}
I20260807 08:29:58.646366 12481 heartbeater.cc:347] Connected to a master server at 127.1.157.124:41513
I20260807 08:29:58.646675 12481 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.646797 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.622842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:56593 (local address 127.1.157.125:44145)
0807 08:29:58.624178 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.624182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.624195 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:29:58.624992 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.624995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.625054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.625653 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.627485 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.628216 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.630236 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.630600 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.632247 (+ 1647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.637704 (+ 5457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.637708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.637710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.637735 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.642269 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.644878 (+ 2609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.644880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.644881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.644955 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.645255 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.645258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.645260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.645472 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.646460 (+ 988us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.646538 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.646648 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1267,"thread_start_us":79,"threads_started":1}
I20260807 08:29:58.646370 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.623210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.1.157.67:56593)
0807 08:29:58.623921 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.623935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.624571 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.625216 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.625219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.625590 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.626313 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.626332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.628340 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.628344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.629982 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.629994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.631961 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.631964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.632048 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.634135 (+ 2087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.634156 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.637544 (+ 3388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.642790 (+ 5246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.642793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.642795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.644708 (+ 1913us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.645049 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.645052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.645054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.645144 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.645590 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.645593 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.645675 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.645748 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.645890 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":42,"spinlock_wait_cycles":4352,"thread_start_us":113,"threads_started":1}
I20260807 08:29:58.646864 12510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38352ac8d21c448f974575b7dc9f7d54. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.646971 12510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: LEARNER
I20260807 08:29:58.647581 12486 heartbeater.cc:347] Connected to a master server at 127.1.157.125:44145
I20260807 08:29:58.647665 12486 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.647716 11754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: NON_VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: true } } }
I20260807 08:29:58.647830 11754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.647950 12486 heartbeater.cc:511] Master 127.1.157.125:44145 requested a full tablet report, sending...
I20260807 08:29:58.647949 12481 heartbeater.cc:511] Master 127.1.157.124:41513 requested a full tablet report, sending...
I20260807 08:29:58.650156 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.623446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:51477)
0807 08:29:58.628542 (+ 5096us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.628555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.628656 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.630977 (+ 2321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.630980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.631181 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:29:58.631301 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.631309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.633260 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.633263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.634041 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.634052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.634894 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.634897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.634984 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.635428 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:29:58.635442 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:29:58.639325 (+ 3883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:29:58.645849 (+ 6524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.645852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.645854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.646064 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.649265 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:29:58.649268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:29:58.649276 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:29:58.649367 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:29:58.649712 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:29:58.649715 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:29:58.649794 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.649874 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.649997 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5035,"thread_start_us":107,"threads_started":1}
I20260807 08:29:58.650292 11523 ts_manager.cc:194] Registered new tserver with Master: 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:29:58.650651 11809 ts_manager.cc:194] Registered new tserver with Master: 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:29:58.652587 11754 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: attempting to promote NON_VOTER f472cd9446d245b98ade5546adc77d96 to VOTER
I20260807 08:29:58.653564 11754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } }
I20260807 08:29:58.651623 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.623798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:51477 (local address 127.1.157.126:38219)
0807 08:29:58.624340 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.624344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.630699 (+ 6355us) server_negotiation.cc:407] Connection header received
0807 08:29:58.630741 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.630744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.630789 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.630865 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.632357 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.633135 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.634377 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.634687 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.634780 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.639459 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.639463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.639465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.639490 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.645681 (+ 6191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.646207 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.646210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.646212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.649131 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.649451 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.649454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.649456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.649616 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.649688 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.649937 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.651480 (+ 1543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":76,"threads_started":1}
I20260807 08:29:58.654832 11537 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260807 08:29:58.654970 11826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 LEARNER]: Refusing update from remote peer 38352ac8d21c448f974575b7dc9f7d54: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 08:29:58.655179 12512 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:29:58.655392 12511 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 08:29:58.655618 12483 heartbeater.cc:347] Connected to a master server at 127.1.157.126:38219
I20260807 08:29:58.655697 12483 heartbeater.cc:464] Registering TS with master...
I20260807 08:29:58.655849 12483 heartbeater.cc:511] Master 127.1.157.126:38219 requested a full tablet report, sending...
I20260807 08:29:58.656420 10639 ts_manager.cc:194] Re-registered known tserver with Master: 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:29:58.656914 12512 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f472cd9446d245b98ade5546adc77d96 (127.1.157.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.657040 10639 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:51477
I20260807 08:29:58.657538 11537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f472cd9446d245b98ade5546adc77d96 (127.1.157.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.658576 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.658706 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.658839 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.658934 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.658711 11754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.659422 11754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54 [sys.catalog]: This master's current role is: LEADER
I20260807 08:29:58.659610 11826 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f472cd9446d245b98ade5546adc77d96 (127.1.157.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.661561 12510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.661679 12510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:29:58.662755 12525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38352ac8d21c448f974575b7dc9f7d54" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:29:58.662931 12525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:29:58.666379 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.660427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36719 (local address 127.0.0.1:49554)
0807 08:29:58.660651 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.660664 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.660744 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.662786 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.662789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.662813 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.662923 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.662930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.664025 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.664028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.664885 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.664895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.665938 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.666090 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.666114 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.666157 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.666222 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":37,"thread_start_us":96,"threads_started":1}
I20260807 08:29:58.666432 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.661214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39143 (local address 127.0.0.1:59410)
0807 08:29:58.661484 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.661497 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.661577 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.662320 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.662323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.662372 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.662458 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.662465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.663851 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.663855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.665012 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.665023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.665934 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.666077 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.666108 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.666185 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.666257 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":53,"thread_start_us":135,"threads_started":1}
I20260807 08:29:58.666447 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.661590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.1.157.67:39143)
0807 08:29:58.661883 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.661894 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.661914 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:29:58.661973 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.661978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.662158 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.662604 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.662713 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.663731 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665145 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.665828 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665918 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.666045 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.666222 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.666312 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":118,"threads_started":1}
I20260807 08:29:58.666760 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.659862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46013 (local address 127.0.0.1:54186)
0807 08:29:58.660314 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.660327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.660390 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.662749 (+ 2359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.662752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.662781 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.662877 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.662883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.664260 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.664263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.665348 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.665357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.666447 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.666451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.666535 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.666561 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.666613 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.666677 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":6,"spinlock_wait_cycles":9600}
I20260807 08:29:58.667214 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.661648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.1.157.66:36719)
0807 08:29:58.662014 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.662026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.662047 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:29:58.662132 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.662137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.662322 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.663091 (+ 769us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.663211 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.663914 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665004 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.665799 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.666879 (+ 1080us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.667009 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.667048 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.667113 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260807 08:29:58.667793 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.660725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.1.157.65:46013)
0807 08:29:58.662061 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.662075 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.662093 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:29:58.662159 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.662163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.662331 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.663026 (+ 695us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.663158 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.664148 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.665473 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.666299 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.667490 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.667590 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.667628 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.667689 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":166,"threads_started":1}
I20260807 08:29:58.669503 12385 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7939366bb54352a7356b6c5edaa750 (DEFAULT_TABLE table=test-table [id=8d33767fd0e9456092f0f83bf230e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:58.669788 12174 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7939366bb54352a7356b6c5edaa750 (DEFAULT_TABLE table=test-table [id=8d33767fd0e9456092f0f83bf230e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:58.670640 12385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7939366bb54352a7356b6c5edaa750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:58.670845 12174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7939366bb54352a7356b6c5edaa750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:58.670508 11975 tablet_service.cc:1467] Processing CreateTablet for tablet 8e7939366bb54352a7356b6c5edaa750 (DEFAULT_TABLE table=test-table [id=8d33767fd0e9456092f0f83bf230e626]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:29:58.671479 11975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e7939366bb54352a7356b6c5edaa750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:29:58.677181 12537 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Bootstrap starting.
I20260807 08:29:58.678705 12538 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Bootstrap starting.
I20260807 08:29:58.679720 12537 tablet_bootstrap.cc:654] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:58.679837 12539 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Bootstrap starting.
I20260807 08:29:58.680639 12537 log.cc:824] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:58.680806 12538 tablet_bootstrap.cc:654] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:58.681567 12538 log.cc:824] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:58.681844 12539 tablet_bootstrap.cc:654] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:29:58.682627 12539 log.cc:824] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:29:58.682703 12537 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: No bootstrap required, opened a new log
I20260807 08:29:58.682925 12537 ts_tablet_manager.cc:1397] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:29:58.683552 12538 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: No bootstrap required, opened a new log
I20260807 08:29:58.683735 12538 ts_tablet_manager.cc:1397] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:29:58.684265 12539 tablet_bootstrap.cc:492] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: No bootstrap required, opened a new log
I20260807 08:29:58.684433 12539 ts_tablet_manager.cc:1397] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:29:58.687428 12537 raft_consensus.cc:348] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.687688 12537 raft_consensus.cc:374] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:58.687794 12537 raft_consensus.cc:729] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5dac72e18c46f4b94b07a54b42ea7d, State: Initialized, Role: FOLLOWER
I20260807 08:29:58.687853 12539 raft_consensus.cc:348] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.688108 12539 raft_consensus.cc:374] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:58.688176 12539 raft_consensus.cc:729] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f63837b354c40a8afe022d9db8369b2, State: Initialized, Role: FOLLOWER
I20260807 08:29:58.688042 12538 raft_consensus.cc:348] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.688308 12538 raft_consensus.cc:374] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:29:58.688272 12537 consensus_queue.cc:260] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.688380 12538 raft_consensus.cc:729] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24623ec868a04df390a3223e0c079963, State: Initialized, Role: FOLLOWER
I20260807 08:29:58.688561 12539 consensus_queue.cc:260] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.688836 12538 consensus_queue.cc:260] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.689272 12273 heartbeater.cc:503] Master 127.1.157.126:38219 was elected leader, sending a full tablet report...
I20260807 08:29:58.689632 12539 ts_tablet_manager.cc:1428] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 08:29:58.689781 12538 ts_tablet_manager.cc:1428] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:29:58.689815 12483 heartbeater.cc:503] Master 127.1.157.126:38219 was elected leader, sending a full tablet report...
I20260807 08:29:58.691929 12070 heartbeater.cc:503] Master 127.1.157.126:38219 was elected leader, sending a full tablet report...
I20260807 08:29:58.695271 12537 ts_tablet_manager.cc:1428] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Time spent starting tablet: real 0.012s user 0.002s sys 0.004s
W20260807 08:29:58.872277 12487 tablet.cc:2369] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:58.894773 12543 raft_consensus.cc:482] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:29:58.894953 12543 raft_consensus.cc:504] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.896112 12543 leader_election.cc:288] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013), 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:58.899544 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.896545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:44423 (local address 127.1.157.66:36719)
0807 08:29:58.896661 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.896664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.896674 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:29:58.896718 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.896721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.896766 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.896832 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.897094 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.897647 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.898692 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.899060 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.899161 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.899215 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.899368 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.899434 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:29:58.899562 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.896353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36719 (local address 127.1.157.67:44423)
0807 08:29:58.896463 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.896476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.896553 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.896868 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.896871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.896887 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.896990 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.896995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.897766 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.897768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.898578 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.898588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.899162 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.899165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.899263 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.899282 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.899345 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.899415 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:29:58.899749 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.896513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:56909 (local address 127.1.157.65:46013)
0807 08:29:58.896635 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.896639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.896650 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:29:58.896696 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.896698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.896733 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.896795 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:29:58.897074 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.897590 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.898802 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.899200 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.899302 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.899352 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.899584 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.899651 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:29:58.899748 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.896293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46013 (local address 127.1.157.67:56909)
0807 08:29:58.896416 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:58.896430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:29:58.896513 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:29:58.896825 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:29:58.896827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:29:58.896843 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:29:58.896955 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.896960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.897697 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.897701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.898676 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:29:58.898692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.899302 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.899305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:29:58.899471 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.899489 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:29:58.899556 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:29:58.899619 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":37}
W20260807 08:29:58.900031 12279 tablet.cc:2369] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:58.900315 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7939366bb54352a7356b6c5edaa750" candidate_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f63837b354c40a8afe022d9db8369b2" is_pre_election: true
I20260807 08:29:58.900395 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7939366bb54352a7356b6c5edaa750" candidate_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24623ec868a04df390a3223e0c079963" is_pre_election: true
I20260807 08:29:58.900615 12005 raft_consensus.cc:2454] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5dac72e18c46f4b94b07a54b42ea7d in term 0.
I20260807 08:29:58.900612 12200 raft_consensus.cc:2454] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5dac72e18c46f4b94b07a54b42ea7d in term 0.
I20260807 08:29:58.901062 12309 leader_election.cc:302] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24623ec868a04df390a3223e0c079963, 4d5dac72e18c46f4b94b07a54b42ea7d, 8f63837b354c40a8afe022d9db8369b2; no voters:
I20260807 08:29:58.901316 12543 raft_consensus.cc:2781] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:29:58.901394 12543 raft_consensus.cc:482] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:29:58.901445 12543 raft_consensus.cc:3037] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:58.902928 12543 raft_consensus.cc:504] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.903232 12543 leader_election.cc:288] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [CANDIDATE]: Term 1 election: Requested vote from peers 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013), 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719)
I20260807 08:29:58.903539 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7939366bb54352a7356b6c5edaa750" candidate_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24623ec868a04df390a3223e0c079963"
I20260807 08:29:58.903570 12200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e7939366bb54352a7356b6c5edaa750" candidate_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f63837b354c40a8afe022d9db8369b2"
I20260807 08:29:58.903667 12200 raft_consensus.cc:3037] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:58.903651 12005 raft_consensus.cc:3037] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:29:58.904587 12005 raft_consensus.cc:2454] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5dac72e18c46f4b94b07a54b42ea7d in term 1.
I20260807 08:29:58.904636 12200 raft_consensus.cc:2454] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5dac72e18c46f4b94b07a54b42ea7d in term 1.
I20260807 08:29:58.904832 12309 leader_election.cc:302] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24623ec868a04df390a3223e0c079963, 4d5dac72e18c46f4b94b07a54b42ea7d; no voters:
I20260807 08:29:58.904986 12543 raft_consensus.cc:2781] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:29:58.905191 12543 raft_consensus.cc:686] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 LEADER]: Becoming Leader. State: Replica: 4d5dac72e18c46f4b94b07a54b42ea7d, State: Running, Role: LEADER
I20260807 08:29:58.905547 12543 consensus_queue.cc:237] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:29:58.908521 10639 catalog_manager.cc:5495] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67). New cstate: current_term: 1 leader_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } health_report { overall_health: HEALTHY } } }
W20260807 08:29:58.920074 12073 tablet.cc:2369] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:29:58.949397 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.945962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.1.157.67:39143)
0807 08:29:58.946081 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.946085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.946098 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:29:58.946263 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.946266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.946315 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.946420 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:29:58.946634 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.947149 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.948056 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.948268 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.948351 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.948629 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:29:58.949017 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:29:58.949114 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.949233 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.949281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:29:58.954689 12200 raft_consensus.cc:1261] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Refusing update from remote peer 4d5dac72e18c46f4b94b07a54b42ea7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:58.954717 12005 raft_consensus.cc:1261] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Refusing update from remote peer 4d5dac72e18c46f4b94b07a54b42ea7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:29:58.955327 12543 consensus_queue.cc:1035] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:58.955493 12548 consensus_queue.cc:1035] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:29:58.962993 12554 mvcc.cc:205] Tried to move back new op lower bound from 7315830370111787008 to 7315830369919324160. Current Snapshot: MvccSnapshot[applied={T|T < 7315830370111787008}]
I20260807 08:29:58.972675 10639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:58.972815 10639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:29:58.976366 10639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38352ac8d21c448f974575b7dc9f7d54: Sending DeleteTablet for 3 replicas of tablet 8e7939366bb54352a7356b6c5edaa750
I20260807 08:29:58.977125 12174 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7939366bb54352a7356b6c5edaa750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:58 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:49554
I20260807 08:29:58.977535 12385 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7939366bb54352a7356b6c5edaa750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:58 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:59410
I20260807 08:29:58.977640 12565 tablet_replica.cc:330] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: stopping tablet replica
I20260807 08:29:58.977941 12567 tablet_replica.cc:330] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: stopping tablet replica
I20260807 08:29:58.977936 12565 raft_consensus.cc:2229] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:58.978271 12567 raft_consensus.cc:2229] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:29:58.978368 12565 raft_consensus.cc:2258] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:58.978622 11975 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e7939366bb54352a7356b6c5edaa750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:29:58 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:54186
I20260807 08:29:58.978720 12567 raft_consensus.cc:2258] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:58.979919 12570 tablet_replica.cc:330] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: stopping tablet replica
I20260807 08:29:58.979972 12567 ts_tablet_manager.cc:1905] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:58.980273 12570 raft_consensus.cc:2229] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:29:58.980516 12570 raft_consensus.cc:2258] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:29:58.981109 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 10595
I20260807 08:29:58.981899 12565 ts_tablet_manager.cc:1905] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:58.984783 12565 ts_tablet_manager.cc:1918] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:58.985253 12565 log.cc:1196] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/8e7939366bb54352a7356b6c5edaa750
I20260807 08:29:58.985805 12565 ts_tablet_manager.cc:1939] T 8e7939366bb54352a7356b6c5edaa750 P 8f63837b354c40a8afe022d9db8369b2: Deleting consensus metadata
I20260807 08:29:58.985780 12567 ts_tablet_manager.cc:1918] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:58.985885 12567 log.cc:1196] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/8e7939366bb54352a7356b6c5edaa750
I20260807 08:29:58.986316 12567 ts_tablet_manager.cc:1939] T 8e7939366bb54352a7356b6c5edaa750 P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting consensus metadata
I20260807 08:29:58.996327 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.980551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.1.157.125:44145)
0807 08:29:58.980783 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.980788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.985152 (+ 4364us) server_negotiation.cc:407] Connection header received
0807 08:29:58.986036 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.986039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.986094 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.986147 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.988336 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.989118 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.990432 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.990782 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:58.990857 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:58.992777 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:58.992781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:58.992783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:58.992811 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:58.994493 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.994951 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.994955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.994957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.995041 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:58.995348 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:58.995351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:58.995353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:58.995558 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:58.995692 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:58.996046 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 08:29:58.996163 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260807 08:29:58.996773 12570 ts_tablet_manager.cc:1905] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:29:58.999334 12570 ts_tablet_manager.cc:1918] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:29:58.999434 12570 log.cc:1196] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/8e7939366bb54352a7356b6c5edaa750
I20260807 08:29:58.999722 12570 ts_tablet_manager.cc:1939] T 8e7939366bb54352a7356b6c5edaa750 P 24623ec868a04df390a3223e0c079963: Deleting consensus metadata
W20260807 08:29:59.003872 12106 connection.cc:535] server connection from 127.0.0.1:49554 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:29:59.003875 12312 connection.cc:535] server connection from 127.0.0.1:59410 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:29:59.004241 11902 connection.cc:535] server connection from 127.0.0.1:54186 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:29:59.006027 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:58.980667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.1.157.124:41513)
0807 08:29:58.980878 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:58.980883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:58.997400 (+ 16517us) server_negotiation.cc:407] Connection header received
0807 08:29:58.997554 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:58.997557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:58.997600 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:58.997645 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:58.998461 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:58.999375 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.000598 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:59.000958 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.001030 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:59.002592 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:59.002596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:59.002599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:59.002625 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:59.004194 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.004736 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.004739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.004741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.004826 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.005095 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.005099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.005100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.005314 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:59.005425 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:59.005744 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:29:59.005876 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260807 08:29:59.015676 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:59.007580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37234 (local address 127.1.157.124:41513)
0807 08:29:59.007674 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:59.007678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:59.007828 (+ 150us) server_negotiation.cc:407] Connection header received
0807 08:29:59.007974 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:59.007977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:59.008015 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:59.008059 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:59.008954 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:59.009626 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.010433 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:59.010764 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.010861 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:59.012300 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:59.012303 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:59.012306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:59.012329 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:59.013854 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.014281 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.014285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.014286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.014393 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.014913 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.014916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.014918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.015107 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:59.015250 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:59.015434 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 08:29:59.015534 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:29:59.016394 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:59.007694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.1.157.125:44145)
0807 08:29:59.007802 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 08:29:59.007805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:29:59.007815 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:29:59.007841 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:29:59.007843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:29:59.007873 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:29:59.007937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:29:59.009490 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:59.010241 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.011206 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:29:59.011539 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:29:59.011631 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:29:59.012949 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:29:59.012952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:29:59.012954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:29:59.012975 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:29:59.014395 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.014741 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.014744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.014745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.014830 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:29:59.015499 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:29:59.015503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:29:59.015504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:29:59.015697 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:29:59.015787 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:29:59.016123 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 08:29:59.016243 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:29:59.966233 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:59.965846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:37459)
0807 08:29:59.966066 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:59.966141 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
W20260807 08:29:59.966526 12273 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:38219 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:29:59.967928 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:59.967540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:47199)
0807 08:29:59.967756 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:59.967837 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":62,"threads_started":1}
W20260807 08:29:59.968163 12483 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:38219 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:29:59.970187 12584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:29:59.969838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:40587)
0807 08:29:59.970046 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 08:29:59.970113 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":63,"threads_started":1}
W20260807 08:29:59.970443 12070 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:38219 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.476550 12587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38352ac8d21c448f974575b7dc9f7d54)
I20260807 08:30:00.476771 12587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } }
I20260807 08:30:00.477547 12588 raft_consensus.cc:482] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38352ac8d21c448f974575b7dc9f7d54)
I20260807 08:30:00.477660 12588 raft_consensus.cc:504] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } }
I20260807 08:30:00.477797 12587 leader_election.cc:288] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219), f472cd9446d245b98ade5546adc77d96 (127.1.157.124:41513)
I20260807 08:30:00.478485 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.477928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60676)
0807 08:30:00.478262 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.478380 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":52,"threads_started":1}
W20260807 08:30:00.478667 11500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.479302 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.478449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60686)
0807 08:30:00.479128 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.479210 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":55,"threads_started":1}
I20260807 08:30:00.479470 12588 leader_election.cc:288] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219), 98545d0754614e78a216b318544c8fc8 (127.1.157.125:44145)
W20260807 08:30:00.479455 11786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.479956 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.479381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60690)
0807 08:30:00.479840 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.479891 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260807 08:30:00.480051 11500 leader_election.cc:334] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219): Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.480451 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.480176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60692)
0807 08:30:00.480334 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.480381 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":86,"threads_started":1}
W20260807 08:30:00.480545 11786 leader_election.cc:334] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219): Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.482535 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.477928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:41513 (local address 127.0.0.1:37236)
0807 08:30:00.478281 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.478296 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.478430 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.478786 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.478790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.478808 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.478906 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.478912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.479887 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.479890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.481213 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.481220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.482184 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.482186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.482259 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.482280 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.482331 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.482430 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":61,"thread_start_us":73,"threads_started":1}
I20260807 08:30:00.482558 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.478141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.1.157.124:41513)
0807 08:30:00.478331 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.478335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.478371 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:30:00.478528 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.478531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.478577 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.478662 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.479040 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.479773 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.481327 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.482068 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.482170 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.482232 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.482378 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.482447 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":128,"thread_start_us":52,"threads_started":1}
I20260807 08:30:00.483044 11826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98545d0754614e78a216b318544c8fc8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f472cd9446d245b98ade5546adc77d96" is_pre_election: true
I20260807 08:30:00.483225 11826 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98545d0754614e78a216b318544c8fc8 in term 1.
I20260807 08:30:00.483541 11501 leader_election.cc:302] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98545d0754614e78a216b318544c8fc8, f472cd9446d245b98ade5546adc77d96; no voters: 38352ac8d21c448f974575b7dc9f7d54
I20260807 08:30:00.483696 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.478855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:44145 (local address 127.0.0.1:58812)
0807 08:30:00.479658 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.479672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.479772 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.480092 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.480096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.480127 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.480274 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.480281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.481327 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.481331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.482595 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.482606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.483318 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.483321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.483409 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.483428 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.483498 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.483571 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":57,"thread_start_us":63,"threads_started":1}
I20260807 08:30:00.483696 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.479102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.1.157.125:44145)
0807 08:30:00.479470 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.479480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.479744 (+ 264us) server_negotiation.cc:407] Connection header received
0807 08:30:00.479856 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.479859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.479904 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.480382 (+ 478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.480470 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.481216 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.482727 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.483211 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.483305 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.483357 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.483525 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.483589 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
I20260807 08:30:00.483762 12587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 08:30:00.483811 12587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38352ac8d21c448f974575b7dc9f7d54)
I20260807 08:30:00.483850 12587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:30:00.484148 11537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f472cd9446d245b98ade5546adc77d96" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "98545d0754614e78a216b318544c8fc8" is_pre_election: true
I20260807 08:30:00.484848 12587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } }
I20260807 08:30:00.485025 11537 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f472cd9446d245b98ade5546adc77d96 in current term 2: Already voted for candidate 98545d0754614e78a216b318544c8fc8 in this term.
I20260807 08:30:00.485141 12587 leader_election.cc:288] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 election: Requested vote from peers 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219), f472cd9446d245b98ade5546adc77d96 (127.1.157.124:41513)
I20260807 08:30:00.485502 11784 leader_election.cc:302] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f472cd9446d245b98ade5546adc77d96; no voters: 38352ac8d21c448f974575b7dc9f7d54, 98545d0754614e78a216b318544c8fc8
I20260807 08:30:00.485571 11826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98545d0754614e78a216b318544c8fc8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f472cd9446d245b98ade5546adc77d96"
I20260807 08:30:00.485664 12588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:30:00.485872 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.485280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60708)
0807 08:30:00.485763 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.485818 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 08:30:00.486454 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.486242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60722)
0807 08:30:00.486335 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.486405 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:30:00.486543 11500 leader_election.cc:334] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219): Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
I20260807 08:30:00.486792 12588 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:30:00.487686 11826 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98545d0754614e78a216b318544c8fc8 in term 2.
I20260807 08:30:00.488210 11501 leader_election.cc:302] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98545d0754614e78a216b318544c8fc8, f472cd9446d245b98ade5546adc77d96; no voters: 38352ac8d21c448f974575b7dc9f7d54
I20260807 08:30:00.488467 12587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:30:00.488647 12587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [term 2 LEADER]: Becoming Leader. State: Replica: 98545d0754614e78a216b318544c8fc8, State: Running, Role: LEADER
I20260807 08:30:00.488880 12587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } }
I20260807 08:30:00.490254 12603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98545d0754614e78a216b318544c8fc8. Latest consensus state: current_term: 2 leader_uuid: "98545d0754614e78a216b318544c8fc8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:30:00.490378 12603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:00.490643 12605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:00.492946 12605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:00.493300 12605 catalog_manager.cc:1259] Loaded cluster ID: fc29cca629f44c0cb4b812553eca344c
I20260807 08:30:00.493343 12605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:00.493808 12605 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:00.494149 12605 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Loaded TSK: 0
I20260807 08:30:00.494787 12605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:30:00.510425 11522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:30:00.513303 11522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:00.515882 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.515676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60748)
0807 08:30:00.515782 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.515825 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:30:00.515923 11826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [term 2 FOLLOWER]: Refusing update from remote peer 98545d0754614e78a216b318544c8fc8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 08:30:00.516393 12587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 08:30:00.516482 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.516296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.0.0.1:60760)
0807 08:30:00.516388 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.516432 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:30:00.516601 11500 consensus_peers.cc:487] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 -> Peer 38352ac8d21c448f974575b7dc9f7d54 (127.1.157.126:38219): Couldn't send request to peer 38352ac8d21c448f974575b7dc9f7d54. Status: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 08:30:00.517753 12588 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98545d0754614e78a216b318544c8fc8. Latest consensus state: current_term: 2 leader_uuid: "98545d0754614e78a216b318544c8fc8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:30:00.517858 12588 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:30:00.518981 12603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "98545d0754614e78a216b318544c8fc8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:30:00.519078 12603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:00.519464 12588 sys_catalog.cc:455] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "98545d0754614e78a216b318544c8fc8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "38352ac8d21c448f974575b7dc9f7d54" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 38219 } } peers { permanent_uuid: "98545d0754614e78a216b318544c8fc8" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 44145 } attrs { promote: false } } peers { permanent_uuid: "f472cd9446d245b98ade5546adc77d96" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 41513 } attrs { promote: false } } }
I20260807 08:30:00.519584 12588 sys_catalog.cc:458] T 00000000000000000000000000000000 P f472cd9446d245b98ade5546adc77d96 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:30:00.519717 12607 mvcc.cc:205] Tried to move back new op lower bound from 7315830376509054976 to 7315830376404586496. Current Snapshot: MvccSnapshot[applied={T|T < 7315830376509054976 or (T in {7315830376509054976})}]
I20260807 08:30:00.528808 12616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39143 (local address 127.0.0.1:59436)
0807 08:30:00.525448 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.525460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.525547 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.525782 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.525785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.525801 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.525893 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.525899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526782 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.526786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.527632 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.527642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528429 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.528432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.528510 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.528533 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.528599 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.528672 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":22,"thread_start_us":96,"threads_started":1}
I20260807 08:30:00.528810 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.1.157.67:39143)
0807 08:30:00.524998 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.525003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.525535 (+ 532us) server_negotiation.cc:407] Connection header received
0807 08:30:00.525627 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.525630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.525679 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.525763 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.525991 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526643 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.527782 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528333 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.528429 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.528496 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.528620 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.528687 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260807 08:30:00.528978 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36719 (local address 127.0.0.1:49558)
0807 08:30:00.524327 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.524342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.524417 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.525396 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.525399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.525416 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.525497 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.525502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526492 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.526495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.527763 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.527774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528633 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.528637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.528716 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.528743 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.528789 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.528859 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":19}
I20260807 08:30:00.529148 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46013 (local address 127.0.0.1:54196)
0807 08:30:00.524629 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.524641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.524736 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.525670 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.525673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.525688 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.525766 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.525771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526840 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.526843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.527977 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.527985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528823 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.528826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.528907 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.528928 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.528971 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.529029 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":23}
I20260807 08:30:00.529373 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.1.157.66:36719)
0807 08:30:00.524990 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.524995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.525009 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:30:00.525062 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.525065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.525224 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.525387 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.525608 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526393 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.527898 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528504 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.529075 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.529139 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.529177 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.529242 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":100,"threads_started":1}
I20260807 08:30:00.529665 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.524633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.1.157.65:46013)
0807 08:30:00.525020 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.525024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.525037 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:00.525080 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.525083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.525258 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.525868 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.525978 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.526736 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.528101 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.528690 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.529397 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.529459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.529496 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.529562 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":74,"threads_started":1}
I20260807 08:30:00.529639 12385 tablet_service.cc:1467] Processing CreateTablet for tablet 529c92abf1ea4aeba509d8717dc258cc (DEFAULT_TABLE table=test-table [id=3e4712c99322456b8dea8ea3ea4e3302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:00.529888 12385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529c92abf1ea4aeba509d8717dc258cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:00.529942 12174 tablet_service.cc:1467] Processing CreateTablet for tablet 529c92abf1ea4aeba509d8717dc258cc (DEFAULT_TABLE table=test-table [id=3e4712c99322456b8dea8ea3ea4e3302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:00.530171 12174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529c92abf1ea4aeba509d8717dc258cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:00.532179 12618 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Bootstrap starting.
I20260807 08:30:00.532567 12619 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Bootstrap starting.
I20260807 08:30:00.532825 11975 tablet_service.cc:1467] Processing CreateTablet for tablet 529c92abf1ea4aeba509d8717dc258cc (DEFAULT_TABLE table=test-table [id=3e4712c99322456b8dea8ea3ea4e3302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:00.533070 11975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529c92abf1ea4aeba509d8717dc258cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:00.533094 12618 tablet_bootstrap.cc:654] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:00.533427 12619 tablet_bootstrap.cc:654] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:00.534552 12619 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: No bootstrap required, opened a new log
I20260807 08:30:00.534629 12619 ts_tablet_manager.cc:1397] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:00.535073 12619 raft_consensus.cc:348] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.535179 12619 raft_consensus.cc:374] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:00.535208 12619 raft_consensus.cc:729] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f63837b354c40a8afe022d9db8369b2, State: Initialized, Role: FOLLOWER
I20260807 08:30:00.535328 12619 consensus_queue.cc:260] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.535646 12619 ts_tablet_manager.cc:1428] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:00.535799 12624 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963: Bootstrap starting.
I20260807 08:30:00.536640 12624 tablet_bootstrap.cc:654] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:00.536791 12618 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: No bootstrap required, opened a new log
I20260807 08:30:00.536865 12618 ts_tablet_manager.cc:1397] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 08:30:00.537245 12618 raft_consensus.cc:348] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.537344 12618 raft_consensus.cc:374] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:00.537375 12618 raft_consensus.cc:729] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5dac72e18c46f4b94b07a54b42ea7d, State: Initialized, Role: FOLLOWER
I20260807 08:30:00.537514 12618 consensus_queue.cc:260] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.537695 12624 tablet_bootstrap.cc:492] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963: No bootstrap required, opened a new log
I20260807 08:30:00.537776 12624 ts_tablet_manager.cc:1397] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:00.537853 12618 ts_tablet_manager.cc:1428] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:00.537973 12278 heartbeater.cc:503] Master 127.1.157.125:44145 was elected leader, sending a full tablet report...
I20260807 08:30:00.538219 12624 raft_consensus.cc:348] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.538360 12624 raft_consensus.cc:374] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:00.538398 12624 raft_consensus.cc:729] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24623ec868a04df390a3223e0c079963, State: Initialized, Role: FOLLOWER
I20260807 08:30:00.538513 12624 consensus_queue.cc:260] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.538826 12624 ts_tablet_manager.cc:1428] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:00.539821 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.539464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:55085)
0807 08:30:00.539674 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.539730 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260807 08:30:00.540669 12635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.540212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:52415)
0807 08:30:00.540550 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.540600 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":95,"threads_started":1}
I20260807 08:30:00.541066 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.538261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:56755)
0807 08:30:00.540952 (+ 2691us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.540998 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2634,"thread_start_us":108,"threads_started":1}
I20260807 08:30:00.541414 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.536115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:51511)
0807 08:30:00.541299 (+ 5184us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.541345 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5122,"thread_start_us":90,"threads_started":1}
I20260807 08:30:00.541656 12486 heartbeater.cc:503] Master 127.1.157.125:44145 was elected leader, sending a full tablet report...
I20260807 08:30:00.542054 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.541815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:40001)
0807 08:30:00.541950 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.541997 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 08:30:00.542558 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.541983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:51263)
0807 08:30:00.542447 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.542499 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
I20260807 08:30:00.678547 12625 raft_consensus.cc:482] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:00.678678 12625 raft_consensus.cc:504] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.679724 12625 leader_election.cc:288] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013), 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:30:00.683449 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.680029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:59399 (local address 127.1.157.65:46013)
0807 08:30:00.680164 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.680168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.680319 (+ 151us) server_negotiation.cc:407] Connection header received
0807 08:30:00.680433 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.680437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.680486 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.680542 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.680842 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.681362 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.682508 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.682913 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.683013 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.683067 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.683251 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.683324 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:30:00.683460 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.680049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:49437 (local address 127.1.157.67:39143)
0807 08:30:00.680163 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.680167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.680374 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:30:00.680483 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.680486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.680529 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.680604 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:00.680859 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.681400 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.682549 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.682929 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.683035 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.683084 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.683290 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.683357 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:30:00.683463 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.679919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46013 (local address 127.1.157.66:59399)
0807 08:30:00.680226 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.680238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.680331 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.680573 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.680576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.680593 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.680733 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.680739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.681471 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.681474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.682399 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.682408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.683046 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.683050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.683132 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.683157 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.683216 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.683276 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":55,"spinlock_wait_cycles":2944}
I20260807 08:30:00.683480 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.679959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39143 (local address 127.1.157.66:49437)
0807 08:30:00.680266 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.680280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:00.680389 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:00.680627 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:00.680630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:00.680656 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:00.680752 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.680756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.681508 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.681511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.682430 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:00.682440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.683055 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.683059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:00.683174 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.683192 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:30:00.683273 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:30:00.683333 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":30,"spinlock_wait_cycles":896,"thread_start_us":134,"threads_started":1}
I20260807 08:30:00.683856 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529c92abf1ea4aeba509d8717dc258cc" candidate_uuid: "8f63837b354c40a8afe022d9db8369b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24623ec868a04df390a3223e0c079963" is_pre_election: true
I20260807 08:30:00.684000 12005 raft_consensus.cc:2454] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f63837b354c40a8afe022d9db8369b2 in term 0.
I20260807 08:30:00.684160 12413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529c92abf1ea4aeba509d8717dc258cc" candidate_uuid: "8f63837b354c40a8afe022d9db8369b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" is_pre_election: true
I20260807 08:30:00.684379 12413 raft_consensus.cc:2454] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f63837b354c40a8afe022d9db8369b2 in term 0.
I20260807 08:30:00.684368 12106 leader_election.cc:302] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24623ec868a04df390a3223e0c079963, 8f63837b354c40a8afe022d9db8369b2; no voters:
I20260807 08:30:00.684609 12625 raft_consensus.cc:2781] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:00.684679 12625 raft_consensus.cc:482] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:00.684717 12625 raft_consensus.cc:3037] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:00.685712 12625 raft_consensus.cc:504] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.686050 12625 leader_election.cc:288] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [CANDIDATE]: Term 1 election: Requested vote from peers 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013), 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143)
I20260807 08:30:00.686381 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529c92abf1ea4aeba509d8717dc258cc" candidate_uuid: "8f63837b354c40a8afe022d9db8369b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24623ec868a04df390a3223e0c079963"
I20260807 08:30:00.686393 12413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529c92abf1ea4aeba509d8717dc258cc" candidate_uuid: "8f63837b354c40a8afe022d9db8369b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d"
I20260807 08:30:00.686486 12413 raft_consensus.cc:3037] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:00.686497 12005 raft_consensus.cc:3037] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:00.687330 12005 raft_consensus.cc:2454] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f63837b354c40a8afe022d9db8369b2 in term 1.
I20260807 08:30:00.687374 12413 raft_consensus.cc:2454] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f63837b354c40a8afe022d9db8369b2 in term 1.
I20260807 08:30:00.687603 12107 leader_election.cc:302] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24623ec868a04df390a3223e0c079963, 4d5dac72e18c46f4b94b07a54b42ea7d, 8f63837b354c40a8afe022d9db8369b2; no voters:
I20260807 08:30:00.687789 12625 raft_consensus.cc:2781] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:00.687992 12625 raft_consensus.cc:686] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 1 LEADER]: Becoming Leader. State: Replica: 8f63837b354c40a8afe022d9db8369b2, State: Running, Role: LEADER
I20260807 08:30:00.688226 12625 consensus_queue.cc:237] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } }
I20260807 08:30:00.689837 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.689416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:43157)
0807 08:30:00.689709 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.689764 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
I20260807 08:30:00.690377 11520 catalog_manager.cc:5495] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "8f63837b354c40a8afe022d9db8369b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f63837b354c40a8afe022d9db8369b2" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 } health_report { overall_health: UNKNOWN } } }
I20260807 08:30:00.714747 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.711747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.1.157.66:36719)
0807 08:30:00.711856 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:00.711859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:00.711871 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:00.711953 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:00.711955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:00.712002 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:00.712080 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:00.712267 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.712736 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.713536 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:00.713731 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:00.713802 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:00.713982 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:00.714398 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:00.714474 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:00.714608 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:30:00.714653 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:30:00.717293 12005 raft_consensus.cc:1261] T 529c92abf1ea4aeba509d8717dc258cc P 24623ec868a04df390a3223e0c079963 [term 1 FOLLOWER]: Refusing update from remote peer 8f63837b354c40a8afe022d9db8369b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:00.717592 12413 raft_consensus.cc:1261] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Refusing update from remote peer 8f63837b354c40a8afe022d9db8369b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:00.717826 12625 consensus_queue.cc:1035] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24623ec868a04df390a3223e0c079963" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:00.718197 12639 consensus_queue.cc:1035] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d5dac72e18c46f4b94b07a54b42ea7d" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:00.719202 12071 heartbeater.cc:503] Master 127.1.157.125:44145 was elected leader, sending a full tablet report...
I20260807 08:30:00.720222 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.719540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:50265)
0807 08:30:00.720055 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.720118 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439}
I20260807 08:30:00.720968 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.720726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:58901)
0807 08:30:00.720835 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.720892 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 08:30:00.724864 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.724575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.66:45691)
0807 08:30:00.724747 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.724801 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 08:30:00.725665 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.725421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.67:56333)
0807 08:30:00.725553 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.725601 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 08:30:00.730531 12636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:00.730297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:38219 (local address 127.1.157.65:51949)
0807 08:30:00.730418 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.730468 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 08:30:00.730753 11520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:00.730886 11520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:00.733342 12645 mvcc.cc:205] Tried to move back new op lower bound from 7315830377333243904 to 7315830377221201920. Current Snapshot: MvccSnapshot[applied={T|T < 7315830377333243904 or (T in {7315830377333243904})}]
I20260807 08:30:00.736162 11520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98545d0754614e78a216b318544c8fc8: Sending DeleteTablet for 3 replicas of tablet 529c92abf1ea4aeba509d8717dc258cc
I20260807 08:30:00.737083 12174 tablet_service.cc:1514] Processing DeleteTablet for tablet 529c92abf1ea4aeba509d8717dc258cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:00 UTC) from {username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:49558
I20260807 08:30:00.737340 12385 tablet_service.cc:1514] Processing DeleteTablet for tablet 529c92abf1ea4aeba509d8717dc258cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:00 UTC) from {username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:59436
I20260807 08:30:00.737703 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 11750
I20260807 08:30:00.737713 11975 tablet_service.cc:1514] Processing DeleteTablet for tablet 529c92abf1ea4aeba509d8717dc258cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:00 UTC) from {username='oryx', principal='oryx/127.1.157.125@KRBTEST.COM'} at 127.0.0.1:54196
I20260807 08:30:00.745152 12656 tablet_replica.cc:330] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: stopping tablet replica
I20260807 08:30:00.745285 12656 raft_consensus.cc:2229] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:30:00.745556 12656 raft_consensus.cc:2258] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:00.745812 12658 tablet_replica.cc:330] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: stopping tablet replica
I20260807 08:30:00.745921 12658 raft_consensus.cc:2229] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:00.746061 12658 raft_consensus.cc:2258] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:00.746871 12656 ts_tablet_manager.cc:1905] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:00.746903 12658 ts_tablet_manager.cc:1905] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:00.749616 12656 ts_tablet_manager.cc:1918] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:00.749630 12658 ts_tablet_manager.cc:1918] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:00.749694 12658 log.cc:1196] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/529c92abf1ea4aeba509d8717dc258cc
I20260807 08:30:00.749810 12656 log.cc:1196] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/529c92abf1ea4aeba509d8717dc258cc
I20260807 08:30:00.749982 12658 ts_tablet_manager.cc:1939] T 529c92abf1ea4aeba509d8717dc258cc P 4d5dac72e18c46f4b94b07a54b42ea7d: Deleting consensus metadata
I20260807 08:30:00.750174 12656 ts_tablet_manager.cc:1939] T 529c92abf1ea4aeba509d8717dc258cc P 8f63837b354c40a8afe022d9db8369b2: Deleting consensus metadata
I20260807 08:30:00.750706 11500 catalog_manager.cc:4841] TS 4d5dac72e18c46f4b94b07a54b42ea7d (127.1.157.67:39143): tablet 529c92abf1ea4aeba509d8717dc258cc (table test-table [id=3e4712c99322456b8dea8ea3ea4e3302]) successfully deleted
I20260807 08:30:00.750819 11502 catalog_manager.cc:4841] TS 8f63837b354c40a8afe022d9db8369b2 (127.1.157.66:36719): tablet 529c92abf1ea4aeba509d8717dc258cc (table test-table [id=3e4712c99322456b8dea8ea3ea4e3302]) successfully deleted
W20260807 08:30:00.754204 12616 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:00.753965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46013 (local address 127.0.0.1:54204)
0807 08:30:00.754078 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:00.754123 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46013: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 08:30:00.754274 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 12080
W20260807 08:30:00.754439 11499 catalog_manager.cc:4584] TS 24623ec868a04df390a3223e0c079963 (127.1.157.65:46013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 529c92abf1ea4aeba509d8717dc258cc: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46013: connect: Connection reset by peer (error 104)
I20260807 08:30:00.763711 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 12283
I20260807 08:30:00.773978 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 11319
I20260807 08:30:00.782829 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 11481
2026-08-07T08:30:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5477 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:30:00 dist-test-slave-6zbq krb5kdc[12661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:30:00 dist-test-slave-6zbq krb5kdc[12661](info): set up 2 sockets
Aug 07 08:30:00 dist-test-slave-6zbq krb5kdc[12661](info): commencing operation
krb5kdc: starting...
W20260807 08:30:01.922283 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:30:01 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:30:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab.
Entry for principal oryx/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab.
I20260807 08:30:02.030519 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33795
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab
--principal=oryx/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:02.148373 12684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:02.148612 12684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:02.148669 12684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:02.152774 12684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:02.152832 12684 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 08:30:02.152855 12684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:02.152889 12684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:02.152911 12684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:02.152938 12684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:02.158263 12684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33795
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.126.keytab
--principal=oryx/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:02.159171 12684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:02.160068 12684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:02.167737 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:02.167747 12696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:02.167747 12694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.168032 12684 server_base.cc:1034] running on GCE node
I20260807 08:30:02.168550 12684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:02.169500 12684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:02.170645 12684 hybrid_clock.cc:648] HybridClock initialized: now 1786091402170632 us; error 25 us; skew 500 ppm
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:02.173141 12684 init.cc:377] Logged in from keytab as oryx/127.1.157.126@KRBTEST.COM (short username oryx)
I20260807 08:30:02.173897 12684 webserver.cc:460] Webserver started at http://127.1.157.126:45701/ using document root <none> and password file <none>
I20260807 08:30:02.174427 12684 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:02.174491 12684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:02.174660 12684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:02.176565 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "3ec4f1210fc448fda54d8d8508ea4333"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "7a0c0c0af1c6b6a22230d29274b78840"
server_key_iv: "d927e0ac15fdfebb4026be4e92e3ec38"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.177076 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "3ec4f1210fc448fda54d8d8508ea4333"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "7a0c0c0af1c6b6a22230d29274b78840"
server_key_iv: "d927e0ac15fdfebb4026be4e92e3ec38"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.180526 12684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:30:02.182716 12707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.183823 12684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:02.183928 12684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "3ec4f1210fc448fda54d8d8508ea4333"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "7a0c0c0af1c6b6a22230d29274b78840"
server_key_iv: "d927e0ac15fdfebb4026be4e92e3ec38"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.184018 12684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:02.193414 12684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:02.196218 12684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:02.196369 12684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:02.203538 12684 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:33795
I20260807 08:30:02.203578 12796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:33795 every 8 connection(s)
I20260807 08:30:02.204375 12684 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:02.206069 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 12684
I20260807 08:30:02.206192 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:30:02.206491 1653 external_mini_cluster.cc:1428] Setting key 50262620dbec9c88081af8b85e9da26a
I20260807 08:30:02.207023 12797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:02.213459 12797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Bootstrap starting.
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:02.215543 12797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:02.216387 12797 log.cc:824] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:02.218463 12797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: No bootstrap required, opened a new log
I20260807 08:30:02.220868 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.207685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.1.157.126:33795)
0807 08:30:02.208087 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.208104 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.208143 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:30:02.208930 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.208953 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.209332 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.209666 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.210296 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.211813 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.212637 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.213289 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.213438 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.215722 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.215729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.215755 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.215786 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.218117 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.218654 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.218661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.218666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.218808 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.219135 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.219138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.219140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.219497 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.219667 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.219951 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.220236 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":122,"threads_started":1}
I20260807 08:30:02.221349 12797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } }
I20260807 08:30:02.221552 12797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:02.221608 12797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec4f1210fc448fda54d8d8508ea4333, State: Initialized, Role: FOLLOWER
I20260807 08:30:02.221978 12797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } }
I20260807 08:30:02.222098 12797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:02.222162 12797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:02.222265 12797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:02.223114 12797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } }
I20260807 08:30:02.223409 12797 leader_election.cc:302] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ec4f1210fc448fda54d8d8508ea4333; no voters:
I20260807 08:30:02.223657 12797 leader_election.cc:288] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:30:02.223752 12807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:02.223942 12807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [term 1 LEADER]: Becoming Leader. State: Replica: 3ec4f1210fc448fda54d8d8508ea4333, State: Running, Role: LEADER
I20260807 08:30:02.224273 12807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } }
I20260807 08:30:02.224671 12797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:02.225710 12808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ec4f1210fc448fda54d8d8508ea4333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } } }
I20260807 08:30:02.225813 12808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:02.226068 12809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ec4f1210fc448fda54d8d8508ea4333. Latest consensus state: current_term: 1 leader_uuid: "3ec4f1210fc448fda54d8d8508ea4333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec4f1210fc448fda54d8d8508ea4333" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33795 } } }
I20260807 08:30:02.226164 12818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:02.226192 12809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:02.229136 12818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:02.233919 12818 catalog_manager.cc:1347] Generated new cluster ID: a615a57863b8487286fcf6be34a607f2
I20260807 08:30:02.233978 12818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:02.255231 12818 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:30:02.256009 12818 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:02.265326 12818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Generated new TSK 0
I20260807 08:30:02.265862 12818 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
I20260807 08:30:02.294694 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:33795
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:02.402386 12832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:02.402607 12832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:02.402664 12832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:02.406490 12832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:02.406535 12832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:02.406586 12832 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:30:02.406639 12832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:02.411463 12832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:02.412329 12832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:02.413218 12832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:02.421890 12839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:02.421897 12841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.422024 12832 server_base.cc:1034] running on GCE node
W20260807 08:30:02.421895 12843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.422538 12832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:02.423120 12832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:02.424300 12832 hybrid_clock.cc:648] HybridClock initialized: now 1786091402424273 us; error 34 us; skew 500 ppm
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:02.426926 12832 init.cc:377] Logged in from keytab as oryx/127.1.157.65@KRBTEST.COM (short username oryx)
I20260807 08:30:02.427738 12832 webserver.cc:460] Webserver started at http://127.1.157.65:33143/ using document root <none> and password file <none>
I20260807 08:30:02.428215 12832 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:02.428274 12832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:02.428453 12832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:02.430307 12832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "609a8d82d563401480abbc268b68ef85"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "4cc2ee235b503ca2ac368aeb758ab4f2"
server_key_iv: "c40d3ec764e643d7670205aeef73790a"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.430938 12832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "609a8d82d563401480abbc268b68ef85"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "4cc2ee235b503ca2ac368aeb758ab4f2"
server_key_iv: "c40d3ec764e643d7670205aeef73790a"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.434284 12832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:02.436450 12858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.437506 12832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:30:02.437614 12832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "609a8d82d563401480abbc268b68ef85"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "4cc2ee235b503ca2ac368aeb758ab4f2"
server_key_iv: "c40d3ec764e643d7670205aeef73790a"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.437706 12832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:02.452905 12832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:02.455744 12832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:02.455901 12832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:02.457212 12832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:02.457270 12832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.457309 12832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:02.457333 12832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:02.466886 12832 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:42727
I20260807 08:30:02.467126 13024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:42727 every 8 connection(s)
I20260807 08:30:02.467886 12832 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:02.469511 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.457785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:55361 (local address 127.1.157.126:33795)
0807 08:30:02.457932 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.457936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.458733 (+ 797us) server_negotiation.cc:407] Connection header received
0807 08:30:02.459591 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.459595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.459650 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.459723 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.461216 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.461997 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.463232 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.463525 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.463617 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.466079 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.466082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.466084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.466109 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.467662 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.468148 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.468152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.468154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.468259 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.468638 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.468640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.468642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.468816 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.468893 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.469299 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.469396 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:30:02.470237 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.458151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.65:55361)
0807 08:30:02.458578 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.458624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.459403 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.459997 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.460006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.460447 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.461052 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.461087 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.462113 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.462116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.463107 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.463118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.463677 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.463681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.463846 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.464330 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.464345 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.465923 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.467770 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.467775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.467785 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.468053 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.468379 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.468382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.468384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.468550 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.468905 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.468909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.469119 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.469262 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.469482 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":117,"threads_started":1}
I20260807 08:30:02.471030 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 12832
I20260807 08:30:02.471160 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:30:02.471437 1653 external_mini_cluster.cc:1428] Setting key 66e8c409717a1688861ca0c15fa09ed8
I20260807 08:30:02.471662 13025 heartbeater.cc:347] Connected to a master server at 127.1.157.126:33795
I20260807 08:30:02.471813 13025 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:02.472304 13025 heartbeater.cc:511] Master 127.1.157.126:33795 requested a full tablet report, sending...
I20260807 08:30:02.473785 12727 ts_manager.cc:194] Registered new tserver with Master: 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727)
I20260807 08:30:02.474931 12727 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:55361
I20260807 08:30:02.480980 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.473855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:35987 (local address 127.1.157.126:33795)
0807 08:30:02.474995 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.474999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.475012 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:02.475046 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.475048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.475088 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.475162 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.475640 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.476343 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.477113 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.477322 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.477403 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.478391 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.478394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.478396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.478416 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.479598 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.479969 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.479973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.479974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.480041 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.480281 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.480284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.480285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.480432 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.480512 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.480746 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.480841 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064}
I20260807 08:30:02.480995 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.473600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.65:35987)
0807 08:30:02.473861 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.473876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.473962 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.475211 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.475214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.475408 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.475525 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.475532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.476466 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.476469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.477004 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.477011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.477416 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.477419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.477510 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.477892 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.477904 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.478235 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.479695 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.479698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.479700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.479885 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.480119 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.480122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.480124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.480201 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.480517 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.480520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.480602 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.480684 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.480819 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
I20260807 08:30:02.497278 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:33795
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:02.603472 13039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:02.603739 13039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:02.603806 13039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:02.607935 13039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:02.607991 13039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:02.608042 13039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:30:02.608093 13039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:02.613978 13039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:02.614866 13039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:02.615743 13039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:02.624750 13049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:02.624758 13044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.624961 13039 server_base.cc:1034] running on GCE node
W20260807 08:30:02.624760 13046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.625378 13039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:02.625955 13039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:02.627146 13039 hybrid_clock.cc:648] HybridClock initialized: now 1786091402627133 us; error 25 us; skew 500 ppm
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:02.629760 13039 init.cc:377] Logged in from keytab as oryx/127.1.157.66@KRBTEST.COM (short username oryx)
I20260807 08:30:02.630595 13039 webserver.cc:460] Webserver started at http://127.1.157.66:33519/ using document root <none> and password file <none>
I20260807 08:30:02.631110 13039 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:02.631177 13039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:02.631353 13039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:02.633275 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "d668995537f251d1c332dee1c345f211"
server_key_iv: "ec89125d03b6519a20f9959757bd30b4"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.633780 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "d668995537f251d1c332dee1c345f211"
server_key_iv: "ec89125d03b6519a20f9959757bd30b4"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.637280 13039 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:30:02.639513 13058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.640690 13039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:02.640805 13039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "d668995537f251d1c332dee1c345f211"
server_key_iv: "ec89125d03b6519a20f9959757bd30b4"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.640892 13039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:02.667730 13039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:02.670137 13039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:02.670293 13039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:02.671578 13039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:02.671653 13039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.671702 13039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:02.671731 13039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:02.681638 13039 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:39521
I20260807 08:30:02.681684 13242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:39521 every 8 connection(s)
I20260807 08:30:02.682652 13039 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:02.684512 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.672291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46277 (local address 127.1.157.126:33795)
0807 08:30:02.672438 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.672443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.673235 (+ 792us) server_negotiation.cc:407] Connection header received
0807 08:30:02.674186 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.674190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.674247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.674307 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.675570 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.676360 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.677728 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.678060 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.678182 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.680931 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.680935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.680937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.680966 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.682560 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.683088 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.683092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.683094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.683181 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.683535 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.683538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.683540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.683712 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.683802 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.684230 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.684356 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:30:02.685146 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.672665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.66:46277)
0807 08:30:02.673084 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.673129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.673947 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.674670 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.674676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.674979 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.675397 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.675410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.676487 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.676490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.677595 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.677605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.678243 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.678248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.678463 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.678978 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.678999 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.680774 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.682690 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.682696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.682709 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.682971 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.683286 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.683289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.683291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.683444 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.683812 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.683818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.684020 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.684170 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.684458 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
I20260807 08:30:02.686149 13243 heartbeater.cc:347] Connected to a master server at 127.1.157.126:33795
I20260807 08:30:02.686288 13243 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:02.686735 13243 heartbeater.cc:511] Master 127.1.157.126:33795 requested a full tablet report, sending...
I20260807 08:30:02.687698 12729 ts_manager.cc:194] Registered new tserver with Master: 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521)
I20260807 08:30:02.688381 12729 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:46277
I20260807 08:30:02.692894 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 13039
I20260807 08:30:02.693028 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:30:02.693275 1653 external_mini_cluster.cc:1428] Setting key fc42b37f1dd87bfbe918f4cbe96fd83b
I20260807 08:30:02.694731 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.688050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.66:60893)
0807 08:30:02.688732 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.688745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.688821 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.689051 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.689053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.689223 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.689324 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.689330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.690261 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.690264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.690839 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.690848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.691196 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.691198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.691261 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.691606 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.691616 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.691871 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.693326 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.693329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.693330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.693508 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.693768 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.693771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.693772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.693853 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.694241 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.694243 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.694319 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.694472 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.694599 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":110,"threads_started":1}
I20260807 08:30:02.694983 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.688078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:60893 (local address 127.1.157.126:33795)
0807 08:30:02.688187 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.688190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.688813 (+ 623us) server_negotiation.cc:407] Connection header received
0807 08:30:02.688878 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.688881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.688923 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.689000 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.689437 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.690143 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.690925 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.691118 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.691185 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.691975 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.691978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.691980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.691998 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.693226 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.693592 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.693595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.693597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.693673 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.693943 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.693946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.693948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.694146 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.694233 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.694615 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.694720 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for oryx/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
I20260807 08:30:02.718070 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:33795
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:02.827040 13254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:02.827267 13254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:02.827308 13254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:02.831154 13254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:02.831202 13254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:02.831238 13254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 08:30:02.831274 13254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:02.836136 13254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.157.126:33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:02.836971 13254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:02.837831 13254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:02.846668 13266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.846712 13254 server_base.cc:1034] running on GCE node
W20260807 08:30:02.846668 13269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:02.846693 13267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:02.847223 13254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:02.847779 13254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:02.848927 13254 hybrid_clock.cc:648] HybridClock initialized: now 1786091402848908 us; error 32 us; skew 500 ppm
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:02.851500 13254 init.cc:377] Logged in from keytab as oryx/127.1.157.67@KRBTEST.COM (short username oryx)
I20260807 08:30:02.852249 13254 webserver.cc:460] Webserver started at http://127.1.157.67:44405/ using document root <none> and password file <none>
I20260807 08:30:02.852720 13254 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:02.852780 13254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:02.852936 13254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:02.854893 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "bb46b1dc50acd25e62495b696d3102b1"
server_key_iv: "f015f03ac471cbeec648c24e16bd6553"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.855372 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "bb46b1dc50acd25e62495b696d3102b1"
server_key_iv: "f015f03ac471cbeec648c24e16bd6553"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.858702 13254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:30:02.860826 13277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.861909 13254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:02.862004 13254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
format_stamp: "Formatted at 2026-08-07 08:30:02 on dist-test-slave-6zbq"
server_key: "bb46b1dc50acd25e62495b696d3102b1"
server_key_iv: "f015f03ac471cbeec648c24e16bd6553"
server_key_version: "encryptionkey@0"
I20260807 08:30:02.862097 13254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:02.888939 13254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:02.891852 13254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:02.892011 13254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:02.893296 13254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:02.893342 13254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:02.893378 13254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:02.893405 13254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:02.903338 13254 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:35563
I20260807 08:30:02.903568 13447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:35563 every 8 connection(s)
I20260807 08:30:02.904197 13254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:30:02.905887 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.894091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:47513 (local address 127.1.157.126:33795)
0807 08:30:02.894201 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.894205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.895209 (+ 1004us) server_negotiation.cc:407] Connection header received
0807 08:30:02.896223 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.896226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.896286 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.896344 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.897897 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.898706 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.899963 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.900189 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.900281 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.902541 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.902544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.902546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.902571 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.903972 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.904447 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.904450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.904452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.904535 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.904896 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.904900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.904901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.905113 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.905195 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.905626 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.905734 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:30:02.906807 13288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.894611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.67:47513)
0807 08:30:02.895070 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.895110 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.895975 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.896745 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.896754 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.897126 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.897725 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.897745 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.898852 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.898855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.899846 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.899856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.900342 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.900347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.900496 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.900949 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.900964 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.902334 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.904076 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.904081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.904095 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.904354 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.904645 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.904648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.904650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.904802 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.905214 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.905219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.905419 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.905586 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.905895 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":138,"threads_started":1}
I20260807 08:30:02.907850 13451 heartbeater.cc:347] Connected to a master server at 127.1.157.126:33795
I20260807 08:30:02.907981 13451 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:02.908403 13451 heartbeater.cc:511] Master 127.1.157.126:33795 requested a full tablet report, sending...
I20260807 08:30:02.909330 12727 ts_manager.cc:194] Registered new tserver with Master: cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:02.910005 12727 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:47513
I20260807 08:30:02.914299 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 13254
I20260807 08:30:02.914424 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:30:02.914753 1653 external_mini_cluster.cc:1428] Setting key 916c9bf67a86f87448637143471b289b
I20260807 08:30:02.915796 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.909690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.1.157.67:49925)
0807 08:30:02.909960 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.909973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.910060 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.910309 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.910312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.910521 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:02.910631 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.910637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.911564 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.911566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.912086 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.912092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.912468 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.912471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.912530 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.912911 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:02.912925 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:02.913187 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:02.914412 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.914415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.914418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.914635 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.914896 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:02.914900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:02.914902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:02.914983 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:02.915336 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:02.915340 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:02.915419 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.915503 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.915629 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":113,"threads_started":1}
I20260807 08:30:02.915805 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.909740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:49925 (local address 127.1.157.126:33795)
0807 08:30:02.909838 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.909842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.910027 (+ 185us) server_negotiation.cc:407] Connection header received
0807 08:30:02.910155 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.910157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.910194 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.910269 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.910740 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.911456 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.912181 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.912387 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.912464 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.913286 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.913288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.913290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.913304 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.914303 (+ 999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.914724 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.914727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.914728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.914801 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.915066 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.915069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.915071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.915232 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.915316 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.915581 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.915677 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:30:02.916528 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:30:02.938433 12801 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:02.933374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.1.157.126:33795)
0807 08:30:02.933469 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.933473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.933486 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:02.933523 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.933526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.933562 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.933638 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.934210 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.934702 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.935439 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.935631 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.935697 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.938304 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34456: BlockingRecv error: recv got EOF from 127.0.0.1:34456 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 08:30:02 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:02.946564 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.939626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.1.157.126:33795)
0807 08:30:02.939746 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.939749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.939872 (+ 123us) server_negotiation.cc:407] Connection header received
0807 08:30:02.939986 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.939989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.940027 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.940083 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:02.940660 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.941176 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.941929 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.942138 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.942210 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.944138 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:02.944141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:02.944143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:02.944163 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:02.945373 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.945705 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.945707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.945708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.945782 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:02.945973 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:02.945975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:02.945976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:02.946110 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:02.946197 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.946367 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.946452 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:30:02.948783 12729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:30:02.951678 12729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:02.964758 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.960088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.1.157.67:35563)
0807 08:30:02.960463 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.960474 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.960656 (+ 182us) server_negotiation.cc:407] Connection header received
0807 08:30:02.960756 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.960761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.960933 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.961072 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:02.961289 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.961984 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.963610 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964167 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964268 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.964418 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.964542 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.964624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":93,"threads_started":1}
I20260807 08:30:02.964758 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.959733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35563 (local address 127.0.0.1:36654)
0807 08:30:02.960550 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.960559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.960660 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.961094 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.961097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.961112 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:02.961192 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.961198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.962093 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.962096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.963476 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.963488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964275 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.964381 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.964418 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.964517 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.964624 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":52,"thread_start_us":76,"threads_started":1}
I20260807 08:30:02.965335 13495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.959259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42727 (local address 127.0.0.1:60246)
0807 08:30:02.960143 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.960156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.960313 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.960791 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.960796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.960826 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:02.960949 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.960956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.962375 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.962378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.963625 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.963634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964861 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.965039 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.965072 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.965141 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.965208 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":143,"spinlock_wait_cycles":9088,"thread_start_us":119,"threads_started":1}
I20260807 08:30:02.965504 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.959246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.1.157.66:39521)
0807 08:30:02.959640 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.959649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.960026 (+ 377us) server_negotiation.cc:407] Connection header received
0807 08:30:02.960304 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.960310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.960509 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.960630 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:02.961113 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.962252 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.963804 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964616 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964704 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.964830 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.965263 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.965366 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":100,"threads_started":1}
I20260807 08:30:02.965797 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.959617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.1.157.65:42727)
0807 08:30:02.960004 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:02.960017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:02.960240 (+ 223us) server_negotiation.cc:407] Connection header received
0807 08:30:02.960495 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:02.960500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:02.960659 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:02.960786 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:02.961067 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.962220 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.963731 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964520 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964624 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.964781 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:02.965558 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 08:30:02.965660 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":78,"threads_started":1}
I20260807 08:30:02.966084 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:02.959087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.0.0.1:50480)
0807 08:30:02.959895 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:02.959940 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:02.960121 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:02.960811 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:02.960814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:02.960843 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:02.960963 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.960969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.962391 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.962394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.963639 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:02.963649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:02.964880 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:02.964883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:02.965029 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:02.965077 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 08:30:02.965824 (+ 747us) client_negotiation.cc:240] Negotiation successful
0807 08:30:02.965966 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":260,"spinlock_wait_cycles":2304,"thread_start_us":64,"threads_started":1}
I20260807 08:30:02.967777 13347 tablet_service.cc:1467] Processing CreateTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 (DEFAULT_TABLE table=test-table [id=e50a1951758f4a6fa779fdfbcb06175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:02.968638 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b0edee34684b0ea1f24d64ddc9ef01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:02.968444 12935 tablet_service.cc:1467] Processing CreateTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 (DEFAULT_TABLE table=test-table [id=e50a1951758f4a6fa779fdfbcb06175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:02.968717 13136 tablet_service.cc:1467] Processing CreateTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 (DEFAULT_TABLE table=test-table [id=e50a1951758f4a6fa779fdfbcb06175c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:02.969482 12935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b0edee34684b0ea1f24d64ddc9ef01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:02.969604 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b0edee34684b0ea1f24d64ddc9ef01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:02.975167 13498 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Bootstrap starting.
I20260807 08:30:02.976728 13500 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Bootstrap starting.
I20260807 08:30:02.977033 13499 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Bootstrap starting.
I20260807 08:30:02.977269 13498 tablet_bootstrap.cc:654] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:02.978114 13498 log.cc:824] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:02.978770 13500 tablet_bootstrap.cc:654] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:02.979202 13499 tablet_bootstrap.cc:654] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:02.979538 13500 log.cc:824] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:02.979961 13499 log.cc:824] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:02.980713 13498 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: No bootstrap required, opened a new log
I20260807 08:30:02.980901 13498 ts_tablet_manager.cc:1397] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:30:02.981436 13500 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: No bootstrap required, opened a new log
I20260807 08:30:02.981587 13499 tablet_bootstrap.cc:492] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: No bootstrap required, opened a new log
I20260807 08:30:02.981606 13500 ts_tablet_manager.cc:1397] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:30:02.981716 13499 ts_tablet_manager.cc:1397] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:30:02.984400 13499 raft_consensus.cc:348] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.984581 13499 raft_consensus.cc:374] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:02.984611 13499 raft_consensus.cc:729] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Initialized, Role: FOLLOWER
I20260807 08:30:02.984671 13498 raft_consensus.cc:348] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.984902 13498 raft_consensus.cc:374] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:02.984951 13498 raft_consensus.cc:729] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Initialized, Role: FOLLOWER
I20260807 08:30:02.984942 13499 consensus_queue.cc:260] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.985163 13500 raft_consensus.cc:348] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.985378 13500 raft_consensus.cc:374] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:02.985344 13498 consensus_queue.cc:260] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.985425 13500 raft_consensus.cc:729] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Initialized, Role: FOLLOWER
I20260807 08:30:02.985735 13499 ts_tablet_manager.cc:1428] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 08:30:02.985998 13451 heartbeater.cc:503] Master 127.1.157.126:33795 was elected leader, sending a full tablet report...
I20260807 08:30:02.986183 13500 consensus_queue.cc:260] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:02.986234 13498 ts_tablet_manager.cc:1428] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:30:02.986605 13025 heartbeater.cc:503] Master 127.1.157.126:33795 was elected leader, sending a full tablet report...
I20260807 08:30:02.987221 13500 ts_tablet_manager.cc:1428] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:30:02.987717 13243 heartbeater.cc:503] Master 127.1.157.126:33795 was elected leader, sending a full tablet report...
I20260807 08:30:03.005541 13508 raft_consensus.cc:482] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:03.005671 13508 raft_consensus.cc:504] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.006738 13508 leader_election.cc:288] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727)
I20260807 08:30:03.010442 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.007000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:54753 (local address 127.1.157.65:42727)
0807 08:30:03.007128 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.007132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.007145 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:03.007281 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.007284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.007330 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.007414 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.007654 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.008249 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.009361 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.009929 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.010050 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.010105 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.010251 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.010306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:30:03.010454 13288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.006960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42727 (local address 127.1.157.67:54753)
0807 08:30:03.007078 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.007092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.007165 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.007426 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.007429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.007446 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.007558 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.007565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.008346 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.008349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.009241 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.009252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.010038 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.010042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.010142 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.010167 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.010241 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.010309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 08:30:03.011032 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b0edee34684b0ea1f24d64ddc9ef01" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85" is_pre_election: true
I20260807 08:30:03.011127 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.007132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:43603 (local address 127.1.157.66:39521)
0807 08:30:03.007258 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.007262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.007396 (+ 134us) server_negotiation.cc:407] Connection header received
0807 08:30:03.007489 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.007492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.007532 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.007605 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.007858 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.008342 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.010043 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.010638 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.010759 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.010804 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.010969 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.011028 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:30:03.011142 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.006982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.1.157.67:43603)
0807 08:30:03.007298 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.007310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.007420 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.007609 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.007612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.007627 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.007741 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.007768 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.008448 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.008451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.009934 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.009944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.010774 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.010778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.010868 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.010885 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.010948 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.011011 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260807 08:30:03.011292 12965 raft_consensus.cc:2454] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 0.
I20260807 08:30:03.011588 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b0edee34684b0ea1f24d64ddc9ef01" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" is_pre_election: true
I20260807 08:30:03.011730 13281 leader_election.cc:302] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609a8d82d563401480abbc268b68ef85, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:03.011843 13173 raft_consensus.cc:2454] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 0.
I20260807 08:30:03.012001 13508 raft_consensus.cc:2781] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:03.012076 13508 raft_consensus.cc:482] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:03.012120 13508 raft_consensus.cc:3037] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.013036 13508 raft_consensus.cc:504] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.013329 13508 leader_election.cc:288] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 election: Requested vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727)
I20260807 08:30:03.013659 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b0edee34684b0ea1f24d64ddc9ef01" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
I20260807 08:30:03.013659 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b0edee34684b0ea1f24d64ddc9ef01" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85"
I20260807 08:30:03.013773 12965 raft_consensus.cc:3037] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.013773 13173 raft_consensus.cc:3037] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.014972 12965 raft_consensus.cc:2454] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 1.
I20260807 08:30:03.014972 13173 raft_consensus.cc:2454] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 1.
I20260807 08:30:03.015265 13281 leader_election.cc:302] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:03.015465 13508 raft_consensus.cc:2781] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:03.015625 13508 raft_consensus.cc:686] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 LEADER]: Becoming Leader. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Running, Role: LEADER
I20260807 08:30:03.015879 13508 consensus_queue.cc:237] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.018657 12729 catalog_manager.cc:5495] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67). New cstate: current_term: 1 leader_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } health_report { overall_health: HEALTHY } } }
I20260807 08:30:03.041108 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.038075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.1.157.67:35563)
0807 08:30:03.038180 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.038183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.038195 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:03.038351 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.038355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.038404 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.038479 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.038664 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.039128 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.039921 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.040146 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.040222 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.040438 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.040818 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.040909 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.040938 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.040983 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:30:03.046187 12965 raft_consensus.cc:1261] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Refusing update from remote peer cb2ba76f651b4a83aa616c7ee7b7a36f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.046188 13173 raft_consensus.cc:1261] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Refusing update from remote peer cb2ba76f651b4a83aa616c7ee7b7a36f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.046797 13508 consensus_queue.cc:1035] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.046962 13513 consensus_queue.cc:1035] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Connected to new peer: Peer: permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.053447 13517 mvcc.cc:205] Tried to move back new op lower bound from 7315830386871418880 to 7315830386755207168. Current Snapshot: MvccSnapshot[applied={T|T < 7315830386871418880}]
I20260807 08:30:03.055229 13518 mvcc.cc:205] Tried to move back new op lower bound from 7315830386871418880 to 7315830386755207168. Current Snapshot: MvccSnapshot[applied={T|T < 7315830386871418880}]
I20260807 08:30:03.061249 12726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.061398 12726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.063555 12726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Sending DeleteTablet for 3 replicas of tablet 90b0edee34684b0ea1f24d64ddc9ef01
I20260807 08:30:03.064194 13136 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:50480
I20260807 08:30:03.064417 12935 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60246
I20260807 08:30:03.064682 12726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:30:03.064839 13347 tablet_service.cc:1514] Processing DeleteTablet for tablet 90b0edee34684b0ea1f24d64ddc9ef01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36654
W20260807 08:30:03.065071 12726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:03.065267 13527 tablet_replica.cc:330] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: stopping tablet replica
I20260807 08:30:03.065255 13529 tablet_replica.cc:330] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: stopping tablet replica
I20260807 08:30:03.065550 13529 raft_consensus.cc:2229] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:30:03.065577 13527 raft_consensus.cc:2229] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:03.065845 13527 raft_consensus.cc:2258] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.065958 13529 raft_consensus.cc:2258] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.067257 13527 ts_tablet_manager.cc:1905] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.067281 13529 ts_tablet_manager.cc:1905] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.069772 13527 ts_tablet_manager.cc:1918] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.069834 13529 ts_tablet_manager.cc:1918] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.069926 13527 log.cc:1196] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/90b0edee34684b0ea1f24d64ddc9ef01
I20260807 08:30:03.069983 13529 log.cc:1196] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/90b0edee34684b0ea1f24d64ddc9ef01
I20260807 08:30:03.070317 13527 ts_tablet_manager.cc:1939] T 90b0edee34684b0ea1f24d64ddc9ef01 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting consensus metadata
I20260807 08:30:03.070392 13529 ts_tablet_manager.cc:1939] T 90b0edee34684b0ea1f24d64ddc9ef01 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting consensus metadata
I20260807 08:30:03.070981 12935 tablet_service.cc:1467] Processing CreateTablet for tablet a89c2b7f7f7844bda45702090331976c (DEFAULT_TABLE table=test-table [id=6cce9962fee64088b578d9e10c238de4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.071105 13136 tablet_service.cc:1467] Processing CreateTablet for tablet a89c2b7f7f7844bda45702090331976c (DEFAULT_TABLE table=test-table [id=6cce9962fee64088b578d9e10c238de4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.071255 12935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89c2b7f7f7844bda45702090331976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.071331 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89c2b7f7f7844bda45702090331976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.071913 13528 tablet_replica.cc:330] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: stopping tablet replica
I20260807 08:30:03.072146 13528 raft_consensus.cc:2229] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:03.072381 13528 raft_consensus.cc:2258] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.073410 13500 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Bootstrap starting.
I20260807 08:30:03.073480 13499 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Bootstrap starting.
I20260807 08:30:03.074267 13499 tablet_bootstrap.cc:654] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.074304 13500 tablet_bootstrap.cc:654] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.075076 13347 tablet_service.cc:1467] Processing CreateTablet for tablet a89c2b7f7f7844bda45702090331976c (DEFAULT_TABLE table=test-table [id=6cce9962fee64088b578d9e10c238de4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.075315 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89c2b7f7f7844bda45702090331976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.075822 12712 catalog_manager.cc:4841] TS cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563): tablet 90b0edee34684b0ea1f24d64ddc9ef01 (table test-table [id=e50a1951758f4a6fa779fdfbcb06175c]) successfully deleted
I20260807 08:30:03.076574 13500 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: No bootstrap required, opened a new log
I20260807 08:30:03.076642 13500 ts_tablet_manager.cc:1397] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:30:03.076877 13528 ts_tablet_manager.cc:1905] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.077013 13500 raft_consensus.cc:348] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.077102 13500 raft_consensus.cc:374] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.077142 13500 raft_consensus.cc:729] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.077245 13500 consensus_queue.cc:260] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.077419 13498 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Bootstrap starting.
I20260807 08:30:03.077461 13500 ts_tablet_manager.cc:1428] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.078310 13498 tablet_bootstrap.cc:654] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.079576 13528 ts_tablet_manager.cc:1918] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.079661 13528 log.cc:1196] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/90b0edee34684b0ea1f24d64ddc9ef01
I20260807 08:30:03.079947 13528 ts_tablet_manager.cc:1939] T 90b0edee34684b0ea1f24d64ddc9ef01 P 609a8d82d563401480abbc268b68ef85: Deleting consensus metadata
I20260807 08:30:03.080678 13498 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: No bootstrap required, opened a new log
I20260807 08:30:03.080766 13498 ts_tablet_manager.cc:1397] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 08:30:03.080911 12712 catalog_manager.cc:4841] TS 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727): tablet 90b0edee34684b0ea1f24d64ddc9ef01 (table test-table [id=e50a1951758f4a6fa779fdfbcb06175c]) successfully deleted
I20260807 08:30:03.081184 13498 raft_consensus.cc:348] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.081301 13498 raft_consensus.cc:374] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.081336 13498 raft_consensus.cc:729] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.081445 13498 consensus_queue.cc:260] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.081548 12710 catalog_manager.cc:4841] TS 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521): tablet 90b0edee34684b0ea1f24d64ddc9ef01 (table test-table [id=e50a1951758f4a6fa779fdfbcb06175c]) successfully deleted
I20260807 08:30:03.081684 13498 ts_tablet_manager.cc:1428] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 08:30:03.082367 13499 tablet_bootstrap.cc:492] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: No bootstrap required, opened a new log
I20260807 08:30:03.082453 13499 ts_tablet_manager.cc:1397] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260807 08:30:03.082870 13499 raft_consensus.cc:348] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.082978 13499 raft_consensus.cc:374] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.083015 13499 raft_consensus.cc:729] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.083132 13499 consensus_queue.cc:260] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.083395 13499 ts_tablet_manager.cc:1428] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260807 08:30:03.155066 13453 tablet.cc:2369] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:30:03.183848 13244 tablet.cc:2369] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:30:03.219090 13026 tablet.cc:2369] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:30:03.264940 13505 raft_consensus.cc:482] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:03.265075 13505 raft_consensus.cc:504] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.266078 13505 leader_election.cc:288] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.269773 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.266433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40667 (local address 127.1.157.66:39521)
0807 08:30:03.266562 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.266567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.266742 (+ 175us) server_negotiation.cc:407] Connection header received
0807 08:30:03.266931 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.266934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.266975 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.267038 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.267311 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.267789 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.268876 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.269264 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.269365 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.269418 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.269583 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.269654 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:30:03.269820 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.266315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.1.157.65:40667)
0807 08:30:03.266654 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.266666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.266820 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.267071 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.267073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.267091 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.267202 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.267206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.267893 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.267896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.268758 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.268768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.269371 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.269375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.269469 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.269492 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.269559 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.269624 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":21,"thread_start_us":89,"threads_started":1}
I20260807 08:30:03.269820 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.266646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:55983 (local address 127.1.157.67:35563)
0807 08:30:03.266787 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.266791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.266803 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:03.266852 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.266855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.266902 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.266971 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.267263 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.267762 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.268899 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.269300 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.269406 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.269460 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.269637 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.269706 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:30:03.269837 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.266302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35563 (local address 127.1.157.65:55983)
0807 08:30:03.266588 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.266603 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.266717 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.266990 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.266993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.267009 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.267159 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.267165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.267868 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.267872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.268782 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.268791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.269430 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.269434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.269526 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.269546 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.269612 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.269677 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":58,"thread_start_us":95,"threads_started":1}
I20260807 08:30:03.270445 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89c2b7f7f7844bda45702090331976c" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" is_pre_election: true
I20260807 08:30:03.270429 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89c2b7f7f7844bda45702090331976c" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" is_pre_election: true
I20260807 08:30:03.270572 13173 raft_consensus.cc:2454] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 0.
I20260807 08:30:03.270653 13379 raft_consensus.cc:2454] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 0.
I20260807 08:30:03.270951 12863 leader_election.cc:302] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85; no voters:
I20260807 08:30:03.271193 13505 raft_consensus.cc:2781] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:03.271247 13505 raft_consensus.cc:482] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:03.271284 13505 raft_consensus.cc:3037] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.272334 13505 raft_consensus.cc:504] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.272636 13505 leader_election.cc:288] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 election: Requested vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.272946 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89c2b7f7f7844bda45702090331976c" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
I20260807 08:30:03.272961 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89c2b7f7f7844bda45702090331976c" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
I20260807 08:30:03.273067 13379 raft_consensus.cc:3037] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.273067 13173 raft_consensus.cc:3037] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.274125 13173 raft_consensus.cc:2454] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 1.
I20260807 08:30:03.274142 13379 raft_consensus.cc:2454] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 1.
I20260807 08:30:03.274412 12863 leader_election.cc:302] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:03.274580 13505 raft_consensus.cc:2781] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:03.274773 13505 raft_consensus.cc:686] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 1 LEADER]: Becoming Leader. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Running, Role: LEADER
I20260807 08:30:03.274986 13505 consensus_queue.cc:237] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.276962 12729 catalog_manager.cc:5495] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 609a8d82d563401480abbc268b68ef85 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "609a8d82d563401480abbc268b68ef85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } health_report { overall_health: UNKNOWN } } }
Aug 07 08:30:03 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091402, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.126@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:03.343376 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.335783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33795 (local address 127.0.0.1:34486)
0807 08:30:03.336054 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.336065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.336184 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.336393 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.336396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.336784 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:03.336879 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.336886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.337588 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.337591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.338188 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.338197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.338643 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.338647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.338764 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.339271 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:03.339283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:03.340625 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:03.342097 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:03.342107 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:03.342112 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:03.342334 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:03.342580 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:03.342582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:03.342583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:03.342666 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:03.342952 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:03.342957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:03.343054 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.343141 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.343247 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260807 08:30:03.343529 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.335876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.1.157.126:33795)
0807 08:30:03.336000 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.336005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.336147 (+ 142us) server_negotiation.cc:407] Connection header received
0807 08:30:03.336270 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.336272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.336307 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.336375 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:03.336989 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.337493 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.338295 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.338542 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.338632 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.340744 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:03.340747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:03.340749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:03.340772 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:03.342009 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:03.342440 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:03.342443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:03.342444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:03.342515 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:03.342726 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:03.342728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:03.342729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:03.342883 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:03.342950 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.343250 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.343352 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":45}
I20260807 08:30:03.346853 12729 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:34486:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 08:30:03.347530 12729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:03.352036 12935 tablet_service.cc:1467] Processing CreateTablet for tablet 866d89b719144df3a6fefb9dc8276ce9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e830667cb1043b9a93bac73fcad8104]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:30:03.352082 13136 tablet_service.cc:1467] Processing CreateTablet for tablet 866d89b719144df3a6fefb9dc8276ce9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e830667cb1043b9a93bac73fcad8104]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:30:03.352434 12935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866d89b719144df3a6fefb9dc8276ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.352476 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866d89b719144df3a6fefb9dc8276ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.352357 13347 tablet_service.cc:1467] Processing CreateTablet for tablet 866d89b719144df3a6fefb9dc8276ce9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e830667cb1043b9a93bac73fcad8104]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 08:30:03.352680 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 866d89b719144df3a6fefb9dc8276ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.354725 13499 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: Bootstrap starting.
I20260807 08:30:03.355310 13498 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: Bootstrap starting.
I20260807 08:30:03.355585 13499 tablet_bootstrap.cc:654] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.355602 13500 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: Bootstrap starting.
I20260807 08:30:03.356112 13498 tablet_bootstrap.cc:654] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.356503 13500 tablet_bootstrap.cc:654] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.356889 13499 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: No bootstrap required, opened a new log
I20260807 08:30:03.356966 13499 ts_tablet_manager.cc:1397] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:03.357332 13499 raft_consensus.cc:348] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.357440 13499 raft_consensus.cc:374] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.357474 13499 raft_consensus.cc:729] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.357579 13500 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: No bootstrap required, opened a new log
I20260807 08:30:03.357594 13499 consensus_queue.cc:260] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.357668 13500 ts_tablet_manager.cc:1397] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:03.357779 13498 tablet_bootstrap.cc:492] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: No bootstrap required, opened a new log
I20260807 08:30:03.357851 13498 ts_tablet_manager.cc:1397] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 08:30:03.357889 13499 ts_tablet_manager.cc:1428] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.357846 13505 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.357975 13505 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.358084 13500 raft_consensus.cc:348] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.358198 13500 raft_consensus.cc:374] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.358234 13500 raft_consensus.cc:729] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.358211 13498 raft_consensus.cc:348] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.358299 13498 raft_consensus.cc:374] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.358325 13498 raft_consensus.cc:729] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.358374 13500 consensus_queue.cc:260] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.358457 13498 consensus_queue.cc:260] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.358647 13500 ts_tablet_manager.cc:1428] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.358693 13508 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.358819 13508 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.358628 13510 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.358978 13510 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.358705 13498 ts_tablet_manager.cc:1428] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.480865 13510 raft_consensus.cc:482] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:03.481003 13510 raft_consensus.cc:504] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.482187 13510 leader_election.cc:288] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.485625 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.482420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:52709 (local address 127.1.157.65:42727)
0807 08:30:03.482556 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.482561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.482695 (+ 134us) server_negotiation.cc:407] Connection header received
0807 08:30:03.482827 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.482830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.482880 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.482958 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.483217 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.483698 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.484723 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.485117 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.485220 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.485272 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.485438 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.485523 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:30:03.485643 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.482379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42727 (local address 127.1.157.66:52709)
0807 08:30:03.482614 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.482627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.482725 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.482978 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.482981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.482998 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.483120 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.483125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.483799 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.483803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.484602 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.484612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.485224 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.485227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.485326 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.485350 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.485414 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.485480 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260807 08:30:03.485899 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.482870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:59121 (local address 127.1.157.67:35563)
0807 08:30:03.483000 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.483004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.483041 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:30:03.483183 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.483186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.483229 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.483288 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:03.483531 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.484029 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.484976 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.485367 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.485493 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.485544 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.485723 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.485797 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 08:30:03.485893 13554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.482751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35563 (local address 127.1.157.66:59121)
0807 08:30:03.482979 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.482991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.483080 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.483307 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.483309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.483324 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:03.483431 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.483436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.484137 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.484140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.484861 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.484869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.485497 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.485501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.485596 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.485619 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.485694 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.485760 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":59,"threads_started":1}
I20260807 08:30:03.486286 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866d89b719144df3a6fefb9dc8276ce9" candidate_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" is_pre_election: true
I20260807 08:30:03.486370 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866d89b719144df3a6fefb9dc8276ce9" candidate_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85" is_pre_election: true
I20260807 08:30:03.486444 13379 raft_consensus.cc:2454] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25bcf2cfc349cdba8ad44c9d412997 in term 0.
I20260807 08:30:03.486511 12965 raft_consensus.cc:2454] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a25bcf2cfc349cdba8ad44c9d412997 in term 0.
I20260807 08:30:03.486796 13064 leader_election.cc:302] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:03.487002 13510 raft_consensus.cc:2781] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:03.487057 13510 raft_consensus.cc:482] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:03.487085 13510 raft_consensus.cc:3037] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.488116 13510 raft_consensus.cc:504] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.488402 13510 leader_election.cc:288] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [CANDIDATE]: Term 1 election: Requested vote from peers 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.488687 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866d89b719144df3a6fefb9dc8276ce9" candidate_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85"
I20260807 08:30:03.488798 12965 raft_consensus.cc:3037] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.488775 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "866d89b719144df3a6fefb9dc8276ce9" candidate_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
I20260807 08:30:03.488873 13379 raft_consensus.cc:3037] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.489818 12965 raft_consensus.cc:2454] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25bcf2cfc349cdba8ad44c9d412997 in term 1.
I20260807 08:30:03.489945 13379 raft_consensus.cc:2454] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a25bcf2cfc349cdba8ad44c9d412997 in term 1.
I20260807 08:30:03.490103 13064 leader_election.cc:302] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85; no voters:
I20260807 08:30:03.490257 13510 raft_consensus.cc:2781] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:03.490427 13510 raft_consensus.cc:686] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 LEADER]: Becoming Leader. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Running, Role: LEADER
I20260807 08:30:03.490650 13510 consensus_queue.cc:237] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.491743 13557 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: TxnStatusTablet state changed. Reason: New leader 3a25bcf2cfc349cdba8ad44c9d412997. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.491854 13557 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:30:03.492183 13559 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 08:30:03.492372 13559 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 08:30:03.492957 12726 catalog_manager.cc:5495] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } health_report { overall_health: UNKNOWN } } }
I20260807 08:30:03.510295 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.506971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.1.157.66:39521)
0807 08:30:03.507093 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.507097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.507110 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:03.507268 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.507270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.507330 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.507408 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.507636 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.508163 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.509033 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.509238 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.509319 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.509593 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.509960 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.510057 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.510136 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.510181 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:30:03.510329 13544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.506887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.0.0.1:50492)
0807 08:30:03.507021 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.507037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.507138 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.507434 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.507437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.507444 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:30:03.507542 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.507547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.508270 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.508274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.508929 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.508939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.509329 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.509332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.509421 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.509464 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.510047 (+ 583us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.510063 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.510125 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.510208 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:30:03.512971 12965 raft_consensus.cc:1261] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Refusing update from remote peer 3a25bcf2cfc349cdba8ad44c9d412997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.513159 13379 raft_consensus.cc:1261] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Refusing update from remote peer 3a25bcf2cfc349cdba8ad44c9d412997: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.513415 13510 consensus_queue.cc:1035] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.513652 13557 consensus_queue.cc:1035] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.515138 13508 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: TxnStatusTablet state changed. Reason: New leader 3a25bcf2cfc349cdba8ad44c9d412997. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.515141 13505 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: TxnStatusTablet state changed. Reason: New leader 3a25bcf2cfc349cdba8ad44c9d412997. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.515242 13505 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.515245 13508 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.516693 13510 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.516803 13510 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:30:03.516952 13519 mvcc.cc:205] Tried to move back new op lower bound from 7315830388784984064 to 7315830388699840512. Current Snapshot: MvccSnapshot[applied={T|T < 7315830388784984064 or (T in {7315830388784984064})}]
I20260807 08:30:03.517628 13508 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.517724 13508 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P cb2ba76f651b4a83aa616c7ee7b7a36f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.519512 13510 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.519618 13510 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 3a25bcf2cfc349cdba8ad44c9d412997: This TxnStatusTablet replica's current role is: LEADER
I20260807 08:30:03.520159 13505 tablet_replica.cc:439] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } } }
I20260807 08:30:03.520255 13505 tablet_replica.cc:442] T 866d89b719144df3a6fefb9dc8276ce9 P 609a8d82d563401480abbc268b68ef85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 08:30:03.524994 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.521806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.1.157.65:42727)
0807 08:30:03.521919 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.521923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.521935 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:03.521994 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.521997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.522043 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.522121 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.522385 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.522902 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.523688 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.523912 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.523995 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.524290 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.524639 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.524735 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.524829 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.524880 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:30:03.534125 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.531181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:41265 (local address 127.1.157.66:39521)
0807 08:30:03.531306 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.531310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.531523 (+ 213us) server_negotiation.cc:407] Connection header received
0807 08:30:03.531627 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.531630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.531668 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.531723 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.531939 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.532402 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.533205 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.533408 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.533489 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.533750 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.533809 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.533867 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.533971 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.534012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:30:03.534163 13572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.531095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.1.157.65:41265)
0807 08:30:03.531445 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.531455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.531544 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.531744 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.531747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.531754 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:30:03.531850 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.531854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.532501 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.532504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.533100 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.533108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.533506 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.533509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.533591 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.533628 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.533886 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.533892 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.533949 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.534019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":91,"threads_started":1}
I20260807 08:30:03.539546 13379 raft_consensus.cc:1261] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Refusing update from remote peer 609a8d82d563401480abbc268b68ef85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.539758 13173 raft_consensus.cc:1261] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Refusing update from remote peer 609a8d82d563401480abbc268b68ef85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.539950 13536 consensus_queue.cc:1035] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.540094 13505 consensus_queue.cc:1035] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.564606 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.561440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:41099 (local address 127.1.157.65:42727)
0807 08:30:03.561543 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.561547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.561740 (+ 193us) server_negotiation.cc:407] Connection header received
0807 08:30:03.561844 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.561846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.561891 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.561962 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.562210 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.562739 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.563671 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.563892 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.563973 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.564228 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.564302 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.564358 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.564446 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.564487 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:30:03.564640 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.561376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42727 (local address 127.1.157.66:41099)
0807 08:30:03.561668 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:03.561681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:03.561772 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:03.561995 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:03.561998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:03.562005 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:30:03.562104 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.562111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.562848 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.562851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.563561 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:03.563571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.563985 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.563988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:03.564074 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.564114 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.564376 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.564382 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:30:03.564438 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 08:30:03.564510 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
I20260807 08:30:03.591401 12727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.591557 12727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.593624 12727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Sending DeleteTablet for 3 replicas of tablet a89c2b7f7f7844bda45702090331976c
I20260807 08:30:03.594311 13136 tablet_service.cc:1514] Processing DeleteTablet for tablet a89c2b7f7f7844bda45702090331976c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:50480
I20260807 08:30:03.594770 12935 tablet_service.cc:1514] Processing DeleteTablet for tablet a89c2b7f7f7844bda45702090331976c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60246
I20260807 08:30:03.595152 13347 tablet_service.cc:1514] Processing DeleteTablet for tablet a89c2b7f7f7844bda45702090331976c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36654
I20260807 08:30:03.595443 13581 tablet_replica.cc:330] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: stopping tablet replica
I20260807 08:30:03.595566 13581 raft_consensus.cc:2229] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:03.595688 13581 raft_consensus.cc:2258] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.596323 13581 ts_tablet_manager.cc:1905] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.598618 13584 tablet_replica.cc:330] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: stopping tablet replica
I20260807 08:30:03.598757 13584 raft_consensus.cc:2229] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:30:03.599041 13584 raft_consensus.cc:2258] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.599160 13581 ts_tablet_manager.cc:1918] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:30:03.599243 13581 log.cc:1196] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/a89c2b7f7f7844bda45702090331976c
I20260807 08:30:03.599500 13581 ts_tablet_manager.cc:1939] T a89c2b7f7f7844bda45702090331976c P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting consensus metadata
I20260807 08:30:03.599680 13584 ts_tablet_manager.cc:1905] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.600144 12710 catalog_manager.cc:4841] TS 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521): tablet a89c2b7f7f7844bda45702090331976c (table test-table [id=6cce9962fee64088b578d9e10c238de4]) successfully deleted
I20260807 08:30:03.600418 13588 tablet_replica.cc:330] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: stopping tablet replica
I20260807 08:30:03.600543 13588 raft_consensus.cc:2229] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:03.600667 13588 raft_consensus.cc:2258] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.601289 13588 ts_tablet_manager.cc:1905] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.602804 13584 ts_tablet_manager.cc:1918] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:30:03.602887 13584 log.cc:1196] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/a89c2b7f7f7844bda45702090331976c
I20260807 08:30:03.603139 13584 ts_tablet_manager.cc:1939] T a89c2b7f7f7844bda45702090331976c P 609a8d82d563401480abbc268b68ef85: Deleting consensus metadata
I20260807 08:30:03.604192 13588 ts_tablet_manager.cc:1918] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:30:03.604308 13588 log.cc:1196] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/a89c2b7f7f7844bda45702090331976c
I20260807 08:30:03.604564 13588 ts_tablet_manager.cc:1939] T a89c2b7f7f7844bda45702090331976c P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting consensus metadata
I20260807 08:30:03.605077 12712 catalog_manager.cc:4841] TS 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727): tablet a89c2b7f7f7844bda45702090331976c (table test-table [id=6cce9962fee64088b578d9e10c238de4]) successfully deleted
I20260807 08:30:03.605271 12712 catalog_manager.cc:4841] TS cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563): tablet a89c2b7f7f7844bda45702090331976c (table test-table [id=6cce9962fee64088b578d9e10c238de4]) successfully deleted
Aug 07 08:30:03 dist-test-slave-6zbq krb5kdc[12661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 08:30:03 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:03 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:30:03.632503 12801 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:03.624487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.1.157.126:33795)
0807 08:30:03.624607 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.624611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.624913 (+ 302us) server_negotiation.cc:407] Connection header received
0807 08:30:03.625035 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.625038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.625092 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.625177 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:03.626131 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.627061 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.628346 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.628684 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.628798 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.632375 (+ 3577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34496: BlockingRecv error: recv got EOF from 127.0.0.1:34496 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 08:30:03 dist-test-slave-6zbq krb5kdc[12661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM
I20260807 08:30:03.643911 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.634282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.1.157.126:33795)
0807 08:30:03.634454 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.634458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.634862 (+ 404us) server_negotiation.cc:407] Connection header received
0807 08:30:03.634955 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.634957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.635003 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.635082 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:03.635730 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.636479 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.637537 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.637857 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.637949 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.640567 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:03.640571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:03.640574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:03.640601 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:03.642254 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:03.642721 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:03.642725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:03.642727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:03.642819 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:03.643075 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:03.643078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:03.643080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:03.643283 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:03.643405 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.643639 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.643759 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:30:03.646565 12727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:30:03.647028 12727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:03.651976 12935 tablet_service.cc:1467] Processing CreateTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 (DEFAULT_TABLE table=test-table [id=7313a2bcb94541f28bf2efc4a9330472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.652271 12935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dab8d550e72444f95d4f70eaa254b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.654755 13499 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Bootstrap starting.
I20260807 08:30:03.655668 13499 tablet_bootstrap.cc:654] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.656558 13136 tablet_service.cc:1467] Processing CreateTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 (DEFAULT_TABLE table=test-table [id=7313a2bcb94541f28bf2efc4a9330472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.656806 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dab8d550e72444f95d4f70eaa254b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.659404 13500 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Bootstrap starting.
I20260807 08:30:03.660130 13500 tablet_bootstrap.cc:654] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.660753 13347 tablet_service.cc:1467] Processing CreateTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 (DEFAULT_TABLE table=test-table [id=7313a2bcb94541f28bf2efc4a9330472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.661010 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dab8d550e72444f95d4f70eaa254b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.662953 13499 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: No bootstrap required, opened a new log
I20260807 08:30:03.663044 13499 ts_tablet_manager.cc:1397] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260807 08:30:03.663237 13500 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: No bootstrap required, opened a new log
I20260807 08:30:03.663311 13500 ts_tablet_manager.cc:1397] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 08:30:03.663388 13498 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Bootstrap starting.
I20260807 08:30:03.663482 13499 raft_consensus.cc:348] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.663589 13499 raft_consensus.cc:374] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.663632 13499 raft_consensus.cc:729] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.663663 13500 raft_consensus.cc:348] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.663791 13500 raft_consensus.cc:374] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.663749 13499 consensus_queue.cc:260] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.663821 13500 raft_consensus.cc:729] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.663928 13500 consensus_queue.cc:260] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.664011 13499 ts_tablet_manager.cc:1428] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:30:03.664156 13500 ts_tablet_manager.cc:1428] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.664218 13498 tablet_bootstrap.cc:654] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.665400 13498 tablet_bootstrap.cc:492] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: No bootstrap required, opened a new log
I20260807 08:30:03.665483 13498 ts_tablet_manager.cc:1397] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:03.665858 13498 raft_consensus.cc:348] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.665973 13498 raft_consensus.cc:374] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.666018 13498 raft_consensus.cc:729] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.666149 13498 consensus_queue.cc:260] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.666433 13498 ts_tablet_manager.cc:1428] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.762802 13575 raft_consensus.cc:482] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:03.762949 13575 raft_consensus.cc:504] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.763403 13575 leader_election.cc:288] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.763834 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dab8d550e72444f95d4f70eaa254b54" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" is_pre_election: true
I20260807 08:30:03.763985 13173 raft_consensus.cc:2454] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 0.
I20260807 08:30:03.764278 12863 leader_election.cc:302] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85; no voters:
I20260807 08:30:03.765856 13575 raft_consensus.cc:2781] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:03.765935 13575 raft_consensus.cc:482] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:03.765888 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dab8d550e72444f95d4f70eaa254b54" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" is_pre_election: true
I20260807 08:30:03.765981 13575 raft_consensus.cc:3037] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.766021 13379 raft_consensus.cc:2454] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 0.
I20260807 08:30:03.767194 13575 raft_consensus.cc:504] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.767504 13575 leader_election.cc:288] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 election: Requested vote from peers 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521), cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563)
I20260807 08:30:03.767835 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dab8d550e72444f95d4f70eaa254b54" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
I20260807 08:30:03.767841 13379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dab8d550e72444f95d4f70eaa254b54" candidate_uuid: "609a8d82d563401480abbc268b68ef85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f"
I20260807 08:30:03.767943 13173 raft_consensus.cc:3037] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.767951 13379 raft_consensus.cc:3037] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:03.768795 13173 raft_consensus.cc:2454] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 1.
I20260807 08:30:03.768841 13379 raft_consensus.cc:2454] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609a8d82d563401480abbc268b68ef85 in term 1.
I20260807 08:30:03.769066 12863 leader_election.cc:302] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85; no voters:
I20260807 08:30:03.769270 13575 raft_consensus.cc:2781] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:03.769372 13575 raft_consensus.cc:686] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 1 LEADER]: Becoming Leader. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Running, Role: LEADER
I20260807 08:30:03.769505 13575 consensus_queue.cc:237] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.771034 12729 catalog_manager.cc:5495] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 609a8d82d563401480abbc268b68ef85 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "609a8d82d563401480abbc268b68ef85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } health_report { overall_health: UNKNOWN } } }
I20260807 08:30:03.807073 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:03.804107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.1.157.65:42727)
0807 08:30:03.804224 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:03.804228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:03.804241 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:03.804360 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:03.804363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:03.804406 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:03.804465 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:03.804716 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.805273 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.806100 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:03.806321 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:03.806426 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:03.806685 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:03.806753 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:03.806809 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:03.806899 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 08:30:03.806947 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:30:03.808732 13379 raft_consensus.cc:1261] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Refusing update from remote peer 609a8d82d563401480abbc268b68ef85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.808770 13173 raft_consensus.cc:1261] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Refusing update from remote peer 609a8d82d563401480abbc268b68ef85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:03.809103 13505 consensus_queue.cc:1035] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.809252 13575 consensus_queue.cc:1035] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:03.815517 12726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.815652 12726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:03.817250 12726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Sending DeleteTablet for 3 replicas of tablet 4dab8d550e72444f95d4f70eaa254b54
I20260807 08:30:03.817685 12935 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60246
I20260807 08:30:03.817858 13347 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36654
I20260807 08:30:03.818014 13588 tablet_replica.cc:330] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: stopping tablet replica
I20260807 08:30:03.818008 13136 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dab8d550e72444f95d4f70eaa254b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:03 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:50480
I20260807 08:30:03.818137 13588 raft_consensus.cc:2229] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:03.818185 13584 tablet_replica.cc:330] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: stopping tablet replica
I20260807 08:30:03.818261 13588 raft_consensus.cc:2258] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.818279 13584 raft_consensus.cc:2229] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:30:03.818447 13584 raft_consensus.cc:2258] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.818372 12726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 08:30:03.818619 13581 tablet_replica.cc:330] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: stopping tablet replica
I20260807 08:30:03.818730 13581 raft_consensus.cc:2229] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 08:30:03.818734 12726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:03.818840 13581 raft_consensus.cc:2258] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:03.818893 13588 ts_tablet_manager.cc:1905] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.819483 13584 ts_tablet_manager.cc:1905] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.819483 13581 ts_tablet_manager.cc:1905] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:03.820917 13588 ts_tablet_manager.cc:1918] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.820971 13588 log.cc:1196] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/4dab8d550e72444f95d4f70eaa254b54
I20260807 08:30:03.821148 13588 ts_tablet_manager.cc:1939] T 4dab8d550e72444f95d4f70eaa254b54 P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting consensus metadata
I20260807 08:30:03.821677 13581 ts_tablet_manager.cc:1918] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.821674 13584 ts_tablet_manager.cc:1918] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:03.821672 12712 catalog_manager.cc:4841] TS cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563): tablet 4dab8d550e72444f95d4f70eaa254b54 (table test-table [id=7313a2bcb94541f28bf2efc4a9330472]) successfully deleted
I20260807 08:30:03.821743 13581 log.cc:1196] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/4dab8d550e72444f95d4f70eaa254b54
I20260807 08:30:03.821743 13584 log.cc:1196] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/4dab8d550e72444f95d4f70eaa254b54
I20260807 08:30:03.821973 13584 ts_tablet_manager.cc:1939] T 4dab8d550e72444f95d4f70eaa254b54 P 609a8d82d563401480abbc268b68ef85: Deleting consensus metadata
I20260807 08:30:03.821986 13581 ts_tablet_manager.cc:1939] T 4dab8d550e72444f95d4f70eaa254b54 P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting consensus metadata
I20260807 08:30:03.822598 12710 catalog_manager.cc:4841] TS 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521): tablet 4dab8d550e72444f95d4f70eaa254b54 (table test-table [id=7313a2bcb94541f28bf2efc4a9330472]) successfully deleted
I20260807 08:30:03.822999 12712 catalog_manager.cc:4841] TS 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727): tablet 4dab8d550e72444f95d4f70eaa254b54 (table test-table [id=7313a2bcb94541f28bf2efc4a9330472]) successfully deleted
I20260807 08:30:03.825655 12935 tablet_service.cc:1467] Processing CreateTablet for tablet ddec5f6abc614d40913721d39ef8dd6b (DEFAULT_TABLE table=test-table [id=43086d1d63ee49bcab2acbef96d28ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.825786 13347 tablet_service.cc:1467] Processing CreateTablet for tablet ddec5f6abc614d40913721d39ef8dd6b (DEFAULT_TABLE table=test-table [id=43086d1d63ee49bcab2acbef96d28ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.825907 12935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddec5f6abc614d40913721d39ef8dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.826038 13347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddec5f6abc614d40913721d39ef8dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.827586 13499 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: Bootstrap starting.
I20260807 08:30:03.827644 13498 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Bootstrap starting.
I20260807 08:30:03.828049 13136 tablet_service.cc:1467] Processing CreateTablet for tablet ddec5f6abc614d40913721d39ef8dd6b (DEFAULT_TABLE table=test-table [id=43086d1d63ee49bcab2acbef96d28ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:03.828285 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddec5f6abc614d40913721d39ef8dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:03.828359 13498 tablet_bootstrap.cc:654] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.828367 13499 tablet_bootstrap.cc:654] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.829257 13498 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: No bootstrap required, opened a new log
I20260807 08:30:03.829324 13498 ts_tablet_manager.cc:1397] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:03.829645 13498 raft_consensus.cc:348] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.829730 13498 raft_consensus.cc:374] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.829758 13498 raft_consensus.cc:729] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.829869 13498 consensus_queue.cc:260] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.830106 13498 ts_tablet_manager.cc:1428] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:30:03.830152 13500 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Bootstrap starting.
I20260807 08:30:03.830940 13500 tablet_bootstrap.cc:654] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:03.831743 13499 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: No bootstrap required, opened a new log
I20260807 08:30:03.831801 13499 ts_tablet_manager.cc:1397] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 08:30:03.832091 13499 raft_consensus.cc:348] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.832163 13499 raft_consensus.cc:374] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.832182 13499 raft_consensus.cc:729] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609a8d82d563401480abbc268b68ef85, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.832273 13499 consensus_queue.cc:260] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.832511 13499 ts_tablet_manager.cc:1428] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:30:03.832896 13500 tablet_bootstrap.cc:492] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: No bootstrap required, opened a new log
I20260807 08:30:03.832952 13500 ts_tablet_manager.cc:1397] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:30:03.833253 13500 raft_consensus.cc:348] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.833354 13500 raft_consensus.cc:374] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:03.833379 13500 raft_consensus.cc:729] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a25bcf2cfc349cdba8ad44c9d412997, State: Initialized, Role: FOLLOWER
I20260807 08:30:03.833478 13500 consensus_queue.cc:260] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:03.833724 13500 ts_tablet_manager.cc:1428] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:30:04.073074 13508 raft_consensus.cc:482] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:04.073204 13508 raft_consensus.cc:504] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:04.073619 13508 leader_election.cc:288] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727), 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521)
I20260807 08:30:04.074033 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddec5f6abc614d40913721d39ef8dd6b" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" is_pre_election: true
I20260807 08:30:04.074033 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddec5f6abc614d40913721d39ef8dd6b" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85" is_pre_election: true
I20260807 08:30:04.074199 12965 raft_consensus.cc:2454] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 0.
I20260807 08:30:04.074199 13173 raft_consensus.cc:2454] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 0.
I20260807 08:30:04.074555 13281 leader_election.cc:302] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, 609a8d82d563401480abbc268b68ef85, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:04.074752 13508 raft_consensus.cc:2781] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:04.074817 13508 raft_consensus.cc:482] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:04.074843 13508 raft_consensus.cc:3037] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:04.075812 13508 raft_consensus.cc:504] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:04.076116 13508 leader_election.cc:288] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 election: Requested vote from peers 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727), 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521)
I20260807 08:30:04.076452 13173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddec5f6abc614d40913721d39ef8dd6b" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a25bcf2cfc349cdba8ad44c9d412997"
I20260807 08:30:04.076473 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddec5f6abc614d40913721d39ef8dd6b" candidate_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609a8d82d563401480abbc268b68ef85"
I20260807 08:30:04.076579 12965 raft_consensus.cc:3037] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:04.076579 13173 raft_consensus.cc:3037] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:04.077616 12965 raft_consensus.cc:2454] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 1.
I20260807 08:30:04.077616 13173 raft_consensus.cc:2454] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb2ba76f651b4a83aa616c7ee7b7a36f in term 1.
I20260807 08:30:04.077900 13279 leader_election.cc:302] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a25bcf2cfc349cdba8ad44c9d412997, cb2ba76f651b4a83aa616c7ee7b7a36f; no voters:
I20260807 08:30:04.078076 13508 raft_consensus.cc:2781] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:04.078214 13508 raft_consensus.cc:686] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 LEADER]: Becoming Leader. State: Replica: cb2ba76f651b4a83aa616c7ee7b7a36f, State: Running, Role: LEADER
I20260807 08:30:04.078374 13508 consensus_queue.cc:237] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } }
I20260807 08:30:04.079855 12729 catalog_manager.cc:5495] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67). New cstate: current_term: 1 leader_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb2ba76f651b4a83aa616c7ee7b7a36f" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35563 } health_report { overall_health: HEALTHY } } }
I20260807 08:30:04.124616 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.121728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.1.157.67:35563)
0807 08:30:04.121930 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:04.121935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:04.121947 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:04.121988 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:04.121991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:04.122030 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:04.122097 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:04.122356 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.122870 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.123655 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.123884 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.123967 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:04.124205 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:04.124275 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:04.124334 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:04.124431 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 08:30:04.124481 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
I20260807 08:30:04.133183 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.130378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:43569 (local address 127.1.157.66:39521)
0807 08:30:04.130572 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:04.130575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:04.130591 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:30:04.130701 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:04.130704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:04.130743 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:04.130816 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:04.130987 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.131480 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.132206 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.132439 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.132518 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:04.132770 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:04.132837 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:04.132895 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:04.132996 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:30:04.133044 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":54,"threads_started":1}
I20260807 08:30:04.133183 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.130264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:39521 (local address 127.1.157.67:43569)
0807 08:30:04.130545 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:04.130556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:04.130626 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:04.130826 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:04.130828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:04.130833 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:30:04.130920 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:04.130923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.131556 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.131558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:04.132125 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:04.132131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.132523 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.132526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:04.132595 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:04.132634 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:30:04.132912 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:30:04.132919 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 08:30:04.132976 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:30:04.133052 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260807 08:30:04.137840 12965 raft_consensus.cc:1261] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Refusing update from remote peer cb2ba76f651b4a83aa616c7ee7b7a36f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:04.137974 13173 raft_consensus.cc:1261] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Refusing update from remote peer cb2ba76f651b4a83aa616c7ee7b7a36f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:04.138230 13621 consensus_queue.cc:1035] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Connected to new peer: Peer: permanent_uuid: "609a8d82d563401480abbc268b68ef85" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:04.138422 13508 consensus_queue.cc:1035] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a25bcf2cfc349cdba8ad44c9d412997" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:04.152994 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.149992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:36869 (local address 127.1.157.67:35563)
0807 08:30:04.150093 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:04.150097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:04.150254 (+ 157us) server_negotiation.cc:407] Connection header received
0807 08:30:04.150381 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:04.150384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:04.150428 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:04.150487 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:04.150805 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.151296 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.152036 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.152290 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.152378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:04.152598 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:04.152658 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:04.152714 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:04.152842 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 08:30:04.152892 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:30:04.153016 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.149925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35563 (local address 127.1.157.66:36869)
0807 08:30:04.150171 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:04.150183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:04.150279 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:04.150585 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:04.150588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:04.150596 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 08:30:04.150683 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:04.150688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.151380 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.151382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:04.151953 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:04.151961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:04.152393 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:04.152397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:04.152467 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:04.152496 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 08:30:04.152736 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 08:30:04.152751 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 08:30:04.152809 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:30:04.152886 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260807 08:30:04.173026 12729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:04.173153 12729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:04.174860 12729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ec4f1210fc448fda54d8d8508ea4333: Sending DeleteTablet for 3 replicas of tablet ddec5f6abc614d40913721d39ef8dd6b
I20260807 08:30:04.175329 12935 tablet_service.cc:1514] Processing DeleteTablet for tablet ddec5f6abc614d40913721d39ef8dd6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:04 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60246
I20260807 08:30:04.175355 13347 tablet_service.cc:1514] Processing DeleteTablet for tablet ddec5f6abc614d40913721d39ef8dd6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:04 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36654
I20260807 08:30:04.175420 13136 tablet_service.cc:1514] Processing DeleteTablet for tablet ddec5f6abc614d40913721d39ef8dd6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:04 UTC) from {username='oryx', principal='oryx/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:50480
I20260807 08:30:04.175495 13584 tablet_replica.cc:330] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: stopping tablet replica
I20260807 08:30:04.175494 13588 tablet_replica.cc:330] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: stopping tablet replica
I20260807 08:30:04.175570 13581 tablet_replica.cc:330] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: stopping tablet replica
I20260807 08:30:04.175596 13588 raft_consensus.cc:2229] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:30:04.175604 13584 raft_consensus.cc:2229] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:04.175666 13581 raft_consensus.cc:2229] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:04.175693 13584 raft_consensus.cc:2258] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:04.175750 13588 raft_consensus.cc:2258] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:04.175762 13581 raft_consensus.cc:2258] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:04.176205 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 12832
I20260807 08:30:04.176395 13584 ts_tablet_manager.cc:1905] T ddec5f6abc614d40913721d39ef8dd6b P 609a8d82d563401480abbc268b68ef85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:04.176602 13581 ts_tablet_manager.cc:1905] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:04.178674 13581 ts_tablet_manager.cc:1918] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:30:04.178745 13581 log.cc:1196] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/ddec5f6abc614d40913721d39ef8dd6b
I20260807 08:30:04.178970 13581 ts_tablet_manager.cc:1939] T ddec5f6abc614d40913721d39ef8dd6b P 3a25bcf2cfc349cdba8ad44c9d412997: Deleting consensus metadata
I20260807 08:30:04.182195 13588 ts_tablet_manager.cc:1905] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:04.184006 13588 ts_tablet_manager.cc:1918] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 08:30:04.184074 13588 log.cc:1196] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/ddec5f6abc614d40913721d39ef8dd6b
I20260807 08:30:04.184269 13588 ts_tablet_manager.cc:1939] T ddec5f6abc614d40913721d39ef8dd6b P cb2ba76f651b4a83aa616c7ee7b7a36f: Deleting consensus metadata
I20260807 08:30:04.184479 12710 catalog_manager.cc:4841] TS 3a25bcf2cfc349cdba8ad44c9d412997 (127.1.157.66:39521): tablet ddec5f6abc614d40913721d39ef8dd6b (table test-table [id=43086d1d63ee49bcab2acbef96d28ff7]) successfully deleted
I20260807 08:30:04.184825 12712 catalog_manager.cc:4841] TS cb2ba76f651b4a83aa616c7ee7b7a36f (127.1.157.67:35563): tablet ddec5f6abc614d40913721d39ef8dd6b (table test-table [id=43086d1d63ee49bcab2acbef96d28ff7]) successfully deleted
W20260807 08:30:04.190902 12712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:42727 (error 108)
I20260807 08:30:04.191097 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 13039
I20260807 08:30:04.197444 13648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:04.197108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42727 (local address 127.0.0.1:60276)
0807 08:30:04.197303 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:04.197367 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:42727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":57,"threads_started":1}
W20260807 08:30:04.197569 12712 catalog_manager.cc:4584] TS 609a8d82d563401480abbc268b68ef85 (127.1.157.65:42727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ddec5f6abc614d40913721d39ef8dd6b: Network error: Client connection negotiation failed: client connection to 127.1.157.65:42727: connect: Connection refused (error 111)
I20260807 08:30:04.202378 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 13254
I20260807 08:30:04.216068 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 12684
2026-08-07T08:30:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3431 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:30:04 dist-test-slave-6zbq krb5kdc[13653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:30:04 dist-test-slave-6zbq krb5kdc[13653](info): set up 2 sockets
Aug 07 08:30:04 dist-test-slave-6zbq krb5kdc[13653](info): commencing operation
krb5kdc: starting...
W20260807 08:30:05.333051 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:30:05 dist-test-slave-6zbq krb5kdc[13653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:30:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:30:05.443043 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39261
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:05.546833 13674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:05.547053 13674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:05.547091 13674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:05.550881 13674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:05.550922 13674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:05.550940 13674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:05.550961 13674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:05.550974 13674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:05.555861 13674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:39261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:05.556710 13674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:05.557586 13674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:05.565863 13687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:05.565999 13674 server_base.cc:1034] running on GCE node
W20260807 08:30:05.565896 13685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:05.565869 13684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:05.566653 13674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:05.567636 13674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:05.568779 13674 hybrid_clock.cc:648] HybridClock initialized: now 1786091405568759 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T08:30:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1359 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:30:05 dist-test-slave-6zbq krb5kdc[13697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:30:05 dist-test-slave-6zbq krb5kdc[13697](info): set up 2 sockets
Aug 07 08:30:05 dist-test-slave-6zbq krb5kdc[13697](info): commencing operation
krb5kdc: starting...
W20260807 08:30:06.698385 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:30:06 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:30:06.806293 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46713
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:06.909508 13720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:06.909715 13720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:06.909765 13720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:06.913566 13720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:06.913611 13720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:06.913628 13720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:06.913647 13720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:06.913676 13720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:06.918547 13720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:06.919348 13720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:06.920181 13720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:06.927575 13726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:06.927584 13730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:06.927587 13728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:06.927874 13720 server_base.cc:1034] running on GCE node
I20260807 08:30:06.928361 13720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:06.929265 13720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:06.930398 13720 hybrid_clock.cc:648] HybridClock initialized: now 1786091406930379 us; error 30 us; skew 500 ppm
Aug 07 08:30:06 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091406, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:06.932817 13720 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:30:06.933565 13720 webserver.cc:460] Webserver started at http://127.1.157.126:37289/ using document root <none> and password file <none>
I20260807 08:30:06.934005 13720 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:06.934058 13720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:06.934207 13720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:06.936009 13720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "56517b95db464191ba11d296acf35dc5"
format_stamp: "Formatted at 2026-08-07 08:30:06 on dist-test-slave-6zbq"
server_key: "7e23184b88ee52f177f16468ac7edc12"
server_key_iv: "e5dfdcaa0b4cf13cdf87672bd8352b2f"
server_key_version: "encryptionkey@0"
I20260807 08:30:06.936470 13720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "56517b95db464191ba11d296acf35dc5"
format_stamp: "Formatted at 2026-08-07 08:30:06 on dist-test-slave-6zbq"
server_key: "7e23184b88ee52f177f16468ac7edc12"
server_key_iv: "e5dfdcaa0b4cf13cdf87672bd8352b2f"
server_key_version: "encryptionkey@0"
I20260807 08:30:06.939636 13720 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:30:06.941654 13740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:06.942731 13720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:06.942840 13720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "56517b95db464191ba11d296acf35dc5"
format_stamp: "Formatted at 2026-08-07 08:30:06 on dist-test-slave-6zbq"
server_key: "7e23184b88ee52f177f16468ac7edc12"
server_key_iv: "e5dfdcaa0b4cf13cdf87672bd8352b2f"
server_key_version: "encryptionkey@0"
I20260807 08:30:06.942927 13720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:06.966660 13720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:06.968945 13720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:06.969094 13720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:06.975329 13720 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:46713
I20260807 08:30:06.975348 13816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:46713 every 8 connection(s)
I20260807 08:30:06.976143 13720 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:06.978608 13819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:06.981045 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 13720
I20260807 08:30:06.981191 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:30:06.981498 1653 external_mini_cluster.cc:1428] Setting key 54093261a2c478db5ddb4e428654f638
I20260807 08:30:06.984241 13819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Bootstrap starting.
I20260807 08:30:06.986614 13819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:06.987465 13819 log.cc:824] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:06.989534 13819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: No bootstrap required, opened a new log
Aug 07 08:30:06 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:06.992338 13819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:06.992533 13819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:06.992566 13819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56517b95db464191ba11d296acf35dc5, State: Initialized, Role: FOLLOWER
I20260807 08:30:06.992947 13819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:06.993059 13819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:06.993127 13819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:06.993218 13819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:06.993978 13819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:06.994306 13819 leader_election.cc:302] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56517b95db464191ba11d296acf35dc5; no voters:
I20260807 08:30:06.994663 13819 leader_election.cc:288] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:30:06.994743 13830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:06.994928 13830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 56517b95db464191ba11d296acf35dc5, State: Running, Role: LEADER
I20260807 08:30:06.995285 13830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:06.995548 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:06.982851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49776 (local address 127.1.157.126:46713)
0807 08:30:06.983258 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:06.983272 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:06.983305 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:30:06.983882 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:06.983898 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:06.984189 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:06.984465 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:06.985311 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:06.986252 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:06.987417 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:06.987925 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:06.988076 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:06.990566 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:06.990571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:06.990582 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:06.990606 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:06.992573 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:06.993026 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:06.993032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:06.993038 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:06.993204 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:06.993463 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:06.993466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:06.993468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:06.993815 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:06.994003 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:06.994301 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 08:30:06.994610 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
I20260807 08:30:06.995816 13819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:06.996356 13830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56517b95db464191ba11d296acf35dc5. Latest consensus state: current_term: 1 leader_uuid: "56517b95db464191ba11d296acf35dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } } }
I20260807 08:30:06.996555 13830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:06.996758 13834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56517b95db464191ba11d296acf35dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } } }
I20260807 08:30:06.996850 13834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:06.997892 13844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:07.000497 13844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:07.005721 13844 catalog_manager.cc:1347] Generated new cluster ID: 4d384dc504074e3b9463b5892fc444eb
I20260807 08:30:07.005837 13844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:07.052487 13844 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:30:07.053768 13844 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:07.061105 13844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Generated new TSK 0
I20260807 08:30:07.061666 13844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:30:07.093552 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:07.214907 13863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:07.215126 13863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:07.215164 13863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:07.219022 13863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:07.219070 13863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:07.219126 13863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:07.224037 13863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:07.224882 13863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:07.225771 13863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:07.234325 13877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:07.234333 13871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.234431 13863 server_base.cc:1034] running on GCE node
W20260807 08:30:07.234365 13872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.234915 13863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:07.235437 13863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:07.236580 13863 hybrid_clock.cc:648] HybridClock initialized: now 1786091407236555 us; error 37 us; skew 500 ppm
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:07.239153 13863 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:30:07.239900 13863 webserver.cc:460] Webserver started at http://127.1.157.65:39897/ using document root <none> and password file <none>
I20260807 08:30:07.240378 13863 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:07.240440 13863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:07.240621 13863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:07.242492 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "90589daa86374a4083937d6abe902989"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "0155fe32bd249a282e62d32677264fce"
server_key_iv: "ec16e7bbaa7ebad05933fdf009375943"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.243013 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "90589daa86374a4083937d6abe902989"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "0155fe32bd249a282e62d32677264fce"
server_key_iv: "ec16e7bbaa7ebad05933fdf009375943"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.246327 13863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:07.248576 13886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.249719 13863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:30:07.249807 13863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "90589daa86374a4083937d6abe902989"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "0155fe32bd249a282e62d32677264fce"
server_key_iv: "ec16e7bbaa7ebad05933fdf009375943"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.249882 13863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:07.264735 13863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:07.267473 13863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:07.267617 13863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:07.268078 13863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:07.268780 13863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:07.268824 13863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.268859 13863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:07.268879 13863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.276811 13863 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:45951
I20260807 08:30:07.276839 14059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:45951 every 8 connection(s)
I20260807 08:30:07.277555 13863 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:07.279424 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 13863
I20260807 08:30:07.279549 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:30:07.279794 1653 external_mini_cluster.cc:1428] Setting key 2b7fd418970eb0020448f90c5d0c65e4
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:30:07.290760 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.279112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39081 (local address 127.1.157.126:46713)
0807 08:30:07.279247 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:07.279252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:07.280054 (+ 802us) server_negotiation.cc:407] Connection header received
0807 08:30:07.281286 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:07.281289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:07.281341 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:07.281427 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:07.282497 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.283199 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.284687 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.285000 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.285103 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.287288 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:07.287292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:07.287295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:07.287326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:07.288844 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.289302 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.289306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.289308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.289404 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.289690 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.289694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.289695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.289934 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:07.290040 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:07.290480 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 08:30:07.290629 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:30:07.291476 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.279449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:39081)
0807 08:30:07.279901 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:07.279950 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:07.281088 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:07.281557 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:07.281563 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:07.281871 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:07.282321 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.282334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.283325 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.283329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.284559 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.284569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.285127 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.285129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.285266 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.285745 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:07.285768 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:07.287126 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:07.288950 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.288954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.288964 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.289202 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.289485 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.289487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.289488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.289613 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.290035 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:07.290040 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:07.290248 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 08:30:07.290435 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 08:30:07.290719 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:30:07.292686 14061 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46713
I20260807 08:30:07.292932 14061 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:07.293452 14061 heartbeater.cc:511] Master 127.1.157.126:46713 requested a full tablet report, sending...
I20260807 08:30:07.295042 13764 ts_manager.cc:194] Registered new tserver with Master: 90589daa86374a4083937d6abe902989 (127.1.157.65:45951)
I20260807 08:30:07.296649 13764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:39081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:30:07.306586 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:07.410652 14069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:07.410856 14069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:07.410892 14069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:07.414708 14069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:07.414752 14069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:07.414804 14069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:07.419620 14069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:07.420451 14069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:07.421334 14069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:07.429906 14078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.430001 14069 server_base.cc:1034] running on GCE node
W20260807 08:30:07.429912 14079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:07.429917 14082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.430562 14069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:07.431217 14069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:07.432394 14069 hybrid_clock.cc:648] HybridClock initialized: now 1786091407432379 us; error 25 us; skew 500 ppm
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:07.434962 14069 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:30:07.435715 14069 webserver.cc:460] Webserver started at http://127.1.157.66:42563/ using document root <none> and password file <none>
I20260807 08:30:07.436196 14069 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:07.436255 14069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:07.436437 14069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:07.438263 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "4c8cc6e1c8b54bf9aa4dd0edd405516e"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "585e215e04b5b0dbaeb097142a84fc6e"
server_key_iv: "9e3bf4dc4027dc440c4ccf6a07264ab8"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.438809 14069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "4c8cc6e1c8b54bf9aa4dd0edd405516e"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "585e215e04b5b0dbaeb097142a84fc6e"
server_key_iv: "9e3bf4dc4027dc440c4ccf6a07264ab8"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.442189 14069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:07.444360 14096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.445406 14069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:07.445502 14069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "4c8cc6e1c8b54bf9aa4dd0edd405516e"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "585e215e04b5b0dbaeb097142a84fc6e"
server_key_iv: "9e3bf4dc4027dc440c4ccf6a07264ab8"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.445595 14069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:07.464443 14069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:07.466872 14069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:07.467010 14069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:07.467473 14069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:07.468192 14069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:07.468238 14069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.468274 14069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:07.468294 14069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.476414 14069 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:42717
I20260807 08:30:07.476449 14269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:42717 every 8 connection(s)
I20260807 08:30:07.477193 14069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:07.481994 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 14069
I20260807 08:30:07.482115 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:30:07.482420 1653 external_mini_cluster.cc:1428] Setting key 72740b742e9f9af1849abd3e00aed644
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:07.489516 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.478791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:60167 (local address 127.1.157.126:46713)
0807 08:30:07.478925 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:07.478929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:07.479646 (+ 717us) server_negotiation.cc:407] Connection header received
0807 08:30:07.480773 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:07.480776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:07.480821 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:07.480889 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:07.481964 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.482598 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.483667 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.483913 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.483994 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.486388 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:07.486392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:07.486394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:07.486417 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:07.487650 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.488118 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.488121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.488122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.488198 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.488506 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.488509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.488511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.488684 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:07.488769 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:07.489228 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 08:30:07.489363 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:30:07.490198 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.479105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:60167)
0807 08:30:07.479509 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:07.479547 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:07.480594 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:07.481028 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:07.481034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:07.481362 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:07.481840 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.481859 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.482719 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.482722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.483558 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.483567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.484059 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.484064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.484240 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.484738 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:07.484765 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:07.486216 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:07.487769 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.487775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.487787 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.488024 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.488282 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.488285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.488287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.488418 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.488793 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:07.488800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:07.489012 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 08:30:07.489166 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:30:07.489454 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":107,"threads_started":1}
I20260807 08:30:07.491348 14270 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46713
I20260807 08:30:07.491549 14270 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:30:07.491986 14270 heartbeater.cc:511] Master 127.1.157.126:46713 requested a full tablet report, sending...
I20260807 08:30:07.492942 13764 ts_manager.cc:194] Registered new tserver with Master: 4c8cc6e1c8b54bf9aa4dd0edd405516e (127.1.157.66:42717)
I20260807 08:30:07.493660 13764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:60167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:30:07.506799 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:07.610370 14278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:07.610576 14278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:07.610628 14278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:07.614430 14278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:07.614481 14278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:07.614557 14278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:07.619400 14278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:07.620254 14278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:07.621155 14278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:07.629782 14290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:07.629798 14286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:07.629794 14284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.629837 14278 server_base.cc:1034] running on GCE node
I20260807 08:30:07.630398 14278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:07.630916 14278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:07.632076 14278 hybrid_clock.cc:648] HybridClock initialized: now 1786091407632061 us; error 26 us; skew 500 ppm
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:07.634585 14278 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:30:07.635344 14278 webserver.cc:460] Webserver started at http://127.1.157.67:46007/ using document root <none> and password file <none>
I20260807 08:30:07.635816 14278 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:07.635874 14278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:07.636065 14278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:07.637902 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "5daa31a8d7834ca098be4e9200a80a93"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "81375b4cd3b456fe2f83543a13b81fbe"
server_key_iv: "4fc93a1194cc10685888154cd2fd447b"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.638440 14278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "5daa31a8d7834ca098be4e9200a80a93"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "81375b4cd3b456fe2f83543a13b81fbe"
server_key_iv: "4fc93a1194cc10685888154cd2fd447b"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.641746 14278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:30:07.643906 14298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.644930 14278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 08:30:07.645021 14278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "5daa31a8d7834ca098be4e9200a80a93"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "81375b4cd3b456fe2f83543a13b81fbe"
server_key_iv: "4fc93a1194cc10685888154cd2fd447b"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.645114 14278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:07.664528 14278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:07.667019 14278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:07.667178 14278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:07.667711 14278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:07.668467 14278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:07.668511 14278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.668552 14278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:07.668581 14278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.676700 14278 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:44169
I20260807 08:30:07.676731 14478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:44169 every 8 connection(s)
I20260807 08:30:07.677476 14278 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:30:07.681661 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 14278
I20260807 08:30:07.681784 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:30:07.682030 1653 external_mini_cluster.cc:1428] Setting key ab1d7166f99e7cd405a97e1039923594
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:07.689275 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.679031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:56073 (local address 127.1.157.126:46713)
0807 08:30:07.679146 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:07.679149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:07.679875 (+ 726us) server_negotiation.cc:407] Connection header received
0807 08:30:07.681092 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:07.681096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:07.681158 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:07.681231 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:07.682365 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.682976 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.684047 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.684264 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.684351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.686457 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:07.686461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:07.686464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:07.686486 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:07.687495 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.687938 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.687941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.687942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.688016 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:07.688306 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:07.688309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:07.688311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:07.688482 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:07.688566 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:07.689006 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 08:30:07.689125 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:30:07.689927 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:07.679329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:56073)
0807 08:30:07.679731 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:07.679769 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:07.680869 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:07.681403 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:07.681409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:07.681719 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:07.682198 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.682213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.683090 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.683094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.683925 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:07.683935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:07.684391 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:07.684396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:07.684549 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:07.684960 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:07.684975 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:07.686281 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:07.687595 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.687601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.687614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.687845 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.688094 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:07.688097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:07.688099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:07.688221 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:07.688576 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:07.688583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:07.688794 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 08:30:07.688946 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 08:30:07.689225 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260807 08:30:07.690877 14479 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46713
I20260807 08:30:07.691061 14479 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:07.691442 14479 heartbeater.cc:511] Master 127.1.157.126:46713 requested a full tablet report, sending...
I20260807 08:30:07.692299 13764 ts_manager.cc:194] Registered new tserver with Master: 5daa31a8d7834ca098be4e9200a80a93 (127.1.157.67:44169)
I20260807 08:30:07.692811 13764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:56073
I20260807 08:30:07.695647 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
Entry for principal oryx/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab.
WARNING: no policy specified for oryx/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
Entry for principal oryx/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab.
WARNING: no policy specified for oryx/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
Entry for principal oryx/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab.
I20260807 08:30:07.758873 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 13863
I20260807 08:30:07.771432 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 14069
I20260807 08:30:07.783759 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 14278
I20260807 08:30:07.793406 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 13720
I20260807 08:30:07.806587 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46713
--webserver_interface=127.1.157.126
--webserver_port=37289
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:07.911170 14492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:07.911376 14492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:07.911417 14492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:07.915359 14492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:07.915405 14492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:07.915422 14492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:07.915452 14492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:07.915480 14492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:07.920444 14492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:46713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=37289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:07.921286 14492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:07.922132 14492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:07.929579 14498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.929654 14492 server_base.cc:1034] running on GCE node
W20260807 08:30:07.929665 14502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:07.929585 14499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:07.930361 14492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:07.931308 14492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:07.932447 14492 hybrid_clock.cc:648] HybridClock initialized: now 1786091407932430 us; error 34 us; skew 500 ppm
Aug 07 08:30:07 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091407, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:07.934970 14492 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:30:07.935745 14492 webserver.cc:460] Webserver started at http://127.1.157.126:37289/ using document root <none> and password file <none>
I20260807 08:30:07.936228 14492 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:07.936311 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:07.939705 14492 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 08:30:07.941561 14518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:07.942647 14492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 08:30:07.942771 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "56517b95db464191ba11d296acf35dc5"
format_stamp: "Formatted at 2026-08-07 08:30:06 on dist-test-slave-6zbq"
server_key: "7e23184b88ee52f177f16468ac7edc12"
server_key_iv: "e5dfdcaa0b4cf13cdf87672bd8352b2f"
server_key_version: "encryptionkey@0"
I20260807 08:30:07.943225 14492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:07.977311 14492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:07.980105 14492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:07.980264 14492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:07.986943 14492 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:46713
I20260807 08:30:07.986953 14606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:46713 every 8 connection(s)
I20260807 08:30:07.987910 14492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:07.991799 14607 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:30:07.992142 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 14492
I20260807 08:30:07.992532 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:45951
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:46713
--webserver_port=39897
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:30:07.993371 14607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Bootstrap starting.
I20260807 08:30:08.015506 14607 log.cc:824] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:08.023494 14607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:30:08.023880 14607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Bootstrap complete.
I20260807 08:30:08.029457 14607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:08.029963 14607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56517b95db464191ba11d296acf35dc5, State: Initialized, Role: FOLLOWER
I20260807 08:30:08.030526 14607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:08.030681 14607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:08.030750 14607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:08.030843 14607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:30:08.033233 14607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:08.033591 14607 leader_election.cc:302] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56517b95db464191ba11d296acf35dc5; no voters:
I20260807 08:30:08.033948 14607 leader_election.cc:288] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:30:08.034047 14616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:30:08.034313 14616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [term 2 LEADER]: Becoming Leader. State: Replica: 56517b95db464191ba11d296acf35dc5, State: Running, Role: LEADER
I20260807 08:30:08.034668 14616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } }
I20260807 08:30:08.035208 14607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:08.035794 14617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "56517b95db464191ba11d296acf35dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } } }
I20260807 08:30:08.035900 14617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:08.036586 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56517b95db464191ba11d296acf35dc5. Latest consensus state: current_term: 2 leader_uuid: "56517b95db464191ba11d296acf35dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56517b95db464191ba11d296acf35dc5" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46713 } } }
I20260807 08:30:08.036664 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:08.036871 14624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:08.039881 14624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:08.040683 14624 catalog_manager.cc:1259] Loaded cluster ID: 4d384dc504074e3b9463b5892fc444eb
I20260807 08:30:08.040743 14624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:08.043073 14624 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:08.043725 14624 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56517b95db464191ba11d296acf35dc5: Loaded TSK: 0
I20260807 08:30:08.044883 14624 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:30:08.113413 14613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:08.113642 14613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:08.113699 14613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:08.117702 14613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:08.117755 14613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:08.117831 14613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:08.122823 14613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.65.keytab
--principal=oryx/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:45951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=39897
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:08.123682 14613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:08.124560 14613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:08.133352 14647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:08.133343 14649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.133641 14613 server_base.cc:1034] running on GCE node
W20260807 08:30:08.133462 14652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.134116 14613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:08.134864 14613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:08.136070 14613 hybrid_clock.cc:648] HybridClock initialized: now 1786091408136042 us; error 37 us; skew 500 ppm
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091408, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:08.139470 14613 init.cc:377] Logged in from keytab as oryx/127.1.157.65@KRBTEST.COM (short username oryx)
I20260807 08:30:08.140514 14613 webserver.cc:460] Webserver started at http://127.1.157.65:39897/ using document root <none> and password file <none>
I20260807 08:30:08.141175 14613 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:08.141294 14613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:08.146034 14613 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:30:08.148499 14664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.149813 14613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:30:08.149931 14613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "90589daa86374a4083937d6abe902989"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "0155fe32bd249a282e62d32677264fce"
server_key_iv: "ec16e7bbaa7ebad05933fdf009375943"
server_key_version: "encryptionkey@0"
I20260807 08:30:08.150602 14613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:08.171588 14613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:08.174832 14613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:08.174994 14613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:08.175572 14613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:08.176445 14613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:08.176499 14613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.176553 14613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:08.176580 14613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.185542 14613 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:45951
I20260807 08:30:08.185572 14845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:45951 every 8 connection(s)
I20260807 08:30:08.186462 14613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:08.189803 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 14613
I20260807 08:30:08.190244 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:42717
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:46713
--webserver_port=42563
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:30:08.199239 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.188263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:42563 (local address 127.1.157.126:46713)
0807 08:30:08.188673 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.188694 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.188941 (+ 247us) server_negotiation.cc:407] Connection header received
0807 08:30:08.190320 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.190336 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.190715 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.191051 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.192081 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.193005 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.194513 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.195002 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.195117 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.198639 (+ 3522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:42563: BlockingRecv error: recv got EOF from 127.1.157.65:42563 (error 108)
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":99,"threads_started":1}
I20260807 08:30:08.199345 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.188392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:42563)
0807 08:30:08.188804 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.188842 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.189895 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.191112 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.191118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.191438 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.191911 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.191932 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.193142 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.193145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.194395 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.194408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.195169 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.195174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.195371 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.195882 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.195902 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.198468 (+ 2566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":121,"threads_started":1}
W20260807 08:30:08.200173 14848 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:46713 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.157.126:46713: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.205113 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.200540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:34879)
0807 08:30:08.200756 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.200768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.200845 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.201049 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.201052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.201212 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.201303 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.201309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.201979 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.201982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.202553 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.202562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.202981 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.202984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.203068 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.203443 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.203454 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.205003 (+ 1549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158}
W20260807 08:30:08.205219 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.200650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:34879 (local address 127.1.157.126:46713)
0807 08:30:08.200741 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.200744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.200810 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:30:08.200920 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.200923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.200962 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.201025 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.201406 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.201880 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.202666 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.202886 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.202970 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.205118 (+ 2148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:34879: BlockingRecv error: recv got EOF from 127.1.157.65:34879 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.209785 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.205637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:59349)
0807 08:30:08.205741 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.205754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.205826 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.206027 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.206030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.206155 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.206217 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.206222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.206907 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.206910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.207393 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.207400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.207784 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.207787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.207857 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.208206 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.208217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.209682 (+ 1465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:30:08.209941 14848 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:30:08.210047 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.205706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:59349 (local address 127.1.157.126:46713)
0807 08:30:08.205813 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.205816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.205828 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:08.205895 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.205898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.205933 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.205991 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.206323 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.206809 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.207494 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.207693 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.207780 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.209797 (+ 2017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:59349: BlockingRecv error: recv got EOF from 127.1.157.65:59349 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 08:30:08.296965 14852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:08.297175 14852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:08.297214 14852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:08.301151 14852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:08.301198 14852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:08.301263 14852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:08.306249 14852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.66.keytab
--principal=oryx/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:42717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=42563
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:08.307122 14852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:08.307988 14852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:08.316514 14859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:08.316515 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.316702 14852 server_base.cc:1034] running on GCE node
W20260807 08:30:08.316548 14858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.317148 14852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:08.317695 14852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:08.318841 14852 hybrid_clock.cc:648] HybridClock initialized: now 1786091408318825 us; error 25 us; skew 500 ppm
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091408, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:08.321264 14852 init.cc:377] Logged in from keytab as oryx/127.1.157.66@KRBTEST.COM (short username oryx)
I20260807 08:30:08.322016 14852 webserver.cc:460] Webserver started at http://127.1.157.66:42563/ using document root <none> and password file <none>
I20260807 08:30:08.322537 14852 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:08.322618 14852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:08.325996 14852 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:08.327919 14872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.328959 14852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:08.329063 14852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "4c8cc6e1c8b54bf9aa4dd0edd405516e"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "585e215e04b5b0dbaeb097142a84fc6e"
server_key_iv: "9e3bf4dc4027dc440c4ccf6a07264ab8"
server_key_version: "encryptionkey@0"
I20260807 08:30:08.329537 14852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:08.358917 14852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:08.361284 14852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:08.361441 14852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:08.361953 14852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:08.362696 14852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:08.362743 14852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.362802 14852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:08.362828 14852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.370642 14852 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:42717
I20260807 08:30:08.370694 15046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:42717 every 8 connection(s)
I20260807 08:30:08.371592 14852 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:08.376948 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 14852
I20260807 08:30:08.377346 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:44169
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:46713
--webserver_port=46007
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:30:08.381567 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.373119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:54921 (local address 127.1.157.126:46713)
0807 08:30:08.373270 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.373275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.373999 (+ 724us) server_negotiation.cc:407] Connection header received
0807 08:30:08.375158 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.375161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.375213 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.375277 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.376246 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.376787 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.377935 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.378230 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.378335 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.381448 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:54921: BlockingRecv error: recv got EOF from 127.1.157.66:54921 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:30:08.381989 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.373471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:54921)
0807 08:30:08.373865 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.373903 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.374970 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.375411 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.375417 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.375717 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.376141 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.376153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.376886 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.376888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.377816 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.377826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.378391 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.378395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.378527 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.379080 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.379099 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.381291 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
W20260807 08:30:08.382460 15047 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:46713 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.157.126:46713: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.387476 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.382805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:48165)
0807 08:30:08.382945 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.382957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.383021 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.383226 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.383228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.383346 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.383425 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.383429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.384323 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.384325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.384830 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.384836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.385338 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.385340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.385399 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.385795 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.385805 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.387397 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260807 08:30:08.387660 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.382911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:48165 (local address 127.1.157.126:46713)
0807 08:30:08.383027 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.383031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.383047 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:30:08.383088 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.383091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.383135 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.383220 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.383519 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.384235 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.384927 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.385242 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.385342 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.387550 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:48165: BlockingRecv error: recv got EOF from 127.1.157.66:48165 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.392673 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.388115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:47451)
0807 08:30:08.388261 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.388273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.388335 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.388528 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.388530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.388640 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.388701 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.388705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.389591 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.389593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.390064 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.390070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.390618 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.390621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.390672 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.391049 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.391058 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.392598 (+ 1540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 08:30:08.392840 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.388211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47451 (local address 127.1.157.126:46713)
0807 08:30:08.388322 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.388326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.388339 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:08.388397 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.388400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.388440 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.388522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.388798 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.389500 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.390154 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.390522 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.390613 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.392730 (+ 2117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:47451: BlockingRecv error: recv got EOF from 127.1.157.66:47451 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 08:30:08.393014 15047 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:30:08.485714 15053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:08.485911 15053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:08.485953 15053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:08.489791 15053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:08.489838 15053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:08.489903 15053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:08.494763 15053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/krb5kdc/oryx_127.1.157.67.keytab
--principal=oryx/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:44169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=46007
--tserver_master_addrs=127.1.157.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:08.495573 15053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:08.496407 15053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:08.504750 15065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:08.504755 15061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.504798 15053 server_base.cc:1034] running on GCE node
W20260807 08:30:08.504765 15062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:08.505323 15053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:08.505852 15053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:08.506995 15053 hybrid_clock.cc:648] HybridClock initialized: now 1786091408506982 us; error 24 us; skew 500 ppm
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091408, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:08.509349 15053 init.cc:377] Logged in from keytab as oryx/127.1.157.67@KRBTEST.COM (short username oryx)
I20260807 08:30:08.510068 15053 webserver.cc:460] Webserver started at http://127.1.157.67:46007/ using document root <none> and password file <none>
I20260807 08:30:08.510569 15053 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:08.510649 15053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:08.514053 15053 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:08.515873 15079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.516925 15053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:08.517022 15053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "5daa31a8d7834ca098be4e9200a80a93"
format_stamp: "Formatted at 2026-08-07 08:30:07 on dist-test-slave-6zbq"
server_key: "81375b4cd3b456fe2f83543a13b81fbe"
server_key_iv: "4fc93a1194cc10685888154cd2fd447b"
server_key_version: "encryptionkey@0"
I20260807 08:30:08.517467 15053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:08.533008 15053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:08.535540 15053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:08.535702 15053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:08.536221 15053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:08.537081 15053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:08.537132 15053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.537178 15053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:08.537206 15053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:08.545199 15053 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:44169
I20260807 08:30:08.545255 15252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:44169 every 8 connection(s)
I20260807 08:30:08.546092 15053 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.554111 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 15053
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 08:30:08.555990 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.547651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:57221 (local address 127.1.157.126:46713)
0807 08:30:08.547767 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.547771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.548462 (+ 691us) server_negotiation.cc:407] Connection header received
0807 08:30:08.549611 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.549614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.549669 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.549742 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.550842 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.551352 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.552398 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.552620 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.552707 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.555837 (+ 3130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:57221: BlockingRecv error: recv got EOF from 127.1.157.67:57221 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:30:08.556461 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.547957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:57221)
0807 08:30:08.548334 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.548371 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.549421 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.549926 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.549932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.550233 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.550690 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.550703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.551466 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.551469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.552281 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.552291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.552750 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.552763 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.552904 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.553312 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.553326 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.555698 (+ 2372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
W20260807 08:30:08.556887 15253 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:46713 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.157.126:46713: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.562069 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.557264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:43811)
0807 08:30:08.557357 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.557370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.557437 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.557646 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.557648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.557783 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.557860 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.557865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.558667 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.558670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.559402 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.559412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.559921 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.559924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.560012 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.560329 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.560342 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.561964 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 08:30:08.562369 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.557332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:43811 (local address 127.1.157.126:46713)
0807 08:30:08.557433 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.557437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.557449 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:08.557521 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.557524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.557563 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.557622 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.557976 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.558566 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.559526 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.559823 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.559929 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.562077 (+ 2148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:43811: BlockingRecv error: recv got EOF from 127.1.157.67:43811 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:30:08.563511 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.555071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.1.157.126:46713)
0807 08:30:08.555312 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.555316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.555326 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:30:08.555354 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.555356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.555388 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.555458 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.556245 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.556869 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.557941 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.558179 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.558252 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.559598 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:08.559604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:08.559620 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:08.559649 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:08.561351 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:08.561780 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:08.561784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:08.561789 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:08.561897 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:08.562295 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:08.562298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:08.562300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:08.562736 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:08.562886 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:08.563189 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 08:30:08.563402 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":83,"threads_started":1}
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:08 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:08.567206 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:08.562506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:34223)
0807 08:30:08.562598 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:08.562612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:08.562679 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:08.563006 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:08.563008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:08.563148 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:08.563227 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.563233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.564036 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.564038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.564666 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:08.564676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.565089 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.565093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:08.565169 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.565549 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:08.565563 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:08.567127 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 08:30:08.567329 14850 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:08.562701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:34223 (local address 127.1.157.126:46713)
0807 08:30:08.562819 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:08.562823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:08.562833 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:30:08.562863 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:08.562866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:08.562903 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:08.562979 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:08.563341 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.563936 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.564780 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:08.564975 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:08.565075 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:08.567229 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:34223: BlockingRecv error: recv got EOF from 127.1.157.67:34223 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 08:30:08.567373 15253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:09.215341 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:09.210491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:53003)
0807 08:30:09.210735 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:09.210748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:09.210829 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:09.211037 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:09.211039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:09.211202 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:09.211304 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.211309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.211979 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.211981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.212536 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.212543 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.212900 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.212902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.212960 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.213433 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:09.213444 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:09.215222 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":63,"threads_started":1}
W20260807 08:30:09.215497 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:09.210546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:53003 (local address 127.1.157.126:46713)
0807 08:30:09.210803 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:09.210807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:09.210820 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:09.210903 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:09.210905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:09.210946 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:09.211016 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:09.211383 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.211907 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.212613 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.212828 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.212892 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.215365 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:53003: BlockingRecv error: recv got EOF from 127.1.157.65:53003 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:09.398451 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:09.393536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:53535)
0807 08:30:09.393777 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:09.393790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:09.393880 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:09.394079 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:09.394081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:09.394251 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:09.394383 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.394391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.395062 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.395064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.395609 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.395616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.395970 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.395972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.396047 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.396564 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:09.396577 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:09.398314 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":61,"threads_started":1}
W20260807 08:30:09.398595 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:09.393620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:53535 (local address 127.1.157.126:46713)
0807 08:30:09.393752 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:09.393756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:09.393851 (+ 95us) server_negotiation.cc:407] Connection header received
0807 08:30:09.393956 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:09.393958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:09.393999 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:09.394056 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:09.394476 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.394987 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.395690 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.395894 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.395960 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.398477 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:53535: BlockingRecv error: recv got EOF from 127.1.157.66:53535 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:09 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:09.572567 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:09.567860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:46813)
0807 08:30:09.568097 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:09.568112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:09.568204 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:09.568402 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:09.568404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:09.568568 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:09.568698 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.568703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.569362 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.569364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.569938 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:09.569945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.570290 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.570292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:09.570383 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.570799 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:09.570810 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:09.572461 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":64,"threads_started":1}
W20260807 08:30:09.572731 15261 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:09.567928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:46813 (local address 127.1.157.126:46713)
0807 08:30:09.568042 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:09.568045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:09.568169 (+ 124us) server_negotiation.cc:407] Connection header received
0807 08:30:09.568273 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:09.568275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:09.568316 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:09.568379 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:09.568783 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.569286 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.570014 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:09.570217 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:09.570282 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:09.572597 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:46813: BlockingRecv error: recv got EOF from 127.1.157.67:46813 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:10.221324 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:10.216004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:38543)
0807 08:30:10.216263 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:10.216276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:10.216360 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:10.216562 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:10.216564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:10.216732 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:10.216830 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.216837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.217611 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.217615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.218403 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.218419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.218851 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.218854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.218919 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.219371 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:10.219382 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:10.221149 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
W20260807 08:30:10.221449 15270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:10.216065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:38543 (local address 127.1.157.126:46713)
0807 08:30:10.216308 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:10.216311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:10.216323 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:10.216426 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:10.216428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:10.216465 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:10.216536 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:10.216912 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.217487 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.218531 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.218764 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.218838 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.221321 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:38543: BlockingRecv error: recv got EOF from 127.1.157.65:38543 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:10.403831 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:10.399057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:58795)
0807 08:30:10.399328 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:10.399340 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:10.399431 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:10.399638 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:10.399640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:10.399808 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:10.399902 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.399907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.400567 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.400569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.401123 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.401130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.401494 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.401496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.401554 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.402000 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:10.402010 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:10.403705 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
W20260807 08:30:10.403934 15270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:10.399122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58795 (local address 127.1.157.126:46713)
0807 08:30:10.399253 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:10.399257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:10.399398 (+ 141us) server_negotiation.cc:407] Connection header received
0807 08:30:10.399499 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:10.399501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:10.399543 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:10.399604 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:10.399979 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.400493 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.401207 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.401415 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.401486 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.403825 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:58795: BlockingRecv error: recv got EOF from 127.1.157.66:58795 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:10 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:10.582111 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:10.573200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:57885)
0807 08:30:10.573465 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:10.573480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:10.573587 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:10.573826 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:10.573829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:10.574016 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:10.574110 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.574116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.574953 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.574957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.575645 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:10.575659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.576084 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.576088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:10.576170 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.580285 (+ 4115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:10.580297 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:10.581974 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
W20260807 08:30:10.582203 15270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:10.573341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:57885 (local address 127.1.157.126:46713)
0807 08:30:10.573465 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:10.573469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:10.573542 (+ 73us) server_negotiation.cc:407] Connection header received
0807 08:30:10.573671 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:10.573673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:10.573718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:10.573782 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:10.574220 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.574853 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.575751 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:10.575984 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:10.576069 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:10.582107 (+ 6038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:57885: BlockingRecv error: recv got EOF from 127.1.157.67:57885 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:11.227064 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:11.221976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:38093)
0807 08:30:11.222235 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:11.222247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:11.222357 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:11.222566 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:11.222568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:11.222748 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:11.222848 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.222853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.223523 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.223525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.224174 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.224186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.224652 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.224656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.224739 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.225178 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:11.225190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:11.226951 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":81,"threads_started":1}
W20260807 08:30:11.227237 15275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:11.222046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:38093 (local address 127.1.157.126:46713)
0807 08:30:11.222265 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:11.222269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:11.222297 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:30:11.222429 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:11.222431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:11.222471 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:11.222540 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:11.222929 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.223447 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.224299 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.224544 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.224634 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.227086 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:38093: BlockingRecv error: recv got EOF from 127.1.157.65:38093 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":55,"threads_started":1}
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:11.409408 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:11.404519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:47175)
0807 08:30:11.404801 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:11.404818 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:11.404912 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:11.405133 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:11.405135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:11.405305 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:11.405399 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.405405 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.406061 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.406063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.406656 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.406666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.407032 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.407035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.407091 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.407562 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:11.407573 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:11.409297 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":135,"threads_started":1}
W20260807 08:30:11.409518 15275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:11.404590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47175 (local address 127.1.157.126:46713)
0807 08:30:11.404870 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:11.404873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:11.404887 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:30:11.404993 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:11.404996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:11.405038 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:11.405118 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:11.405479 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.405986 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.406745 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.406957 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.407025 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.409419 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:47175: BlockingRecv error: recv got EOF from 127.1.157.66:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:11 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:11.587888 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:11.582962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:48969)
0807 08:30:11.583306 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:11.583321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:11.583432 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:11.583653 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:11.583655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:11.583826 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:11.583921 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.583926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.584622 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.584624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.585182 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:11.585190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.585590 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.585593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:11.585654 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.586104 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:11.586116 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:11.587768 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":84,"threads_started":1}
W20260807 08:30:11.587996 15275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:11.583043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:48969 (local address 127.1.157.126:46713)
0807 08:30:11.583205 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:11.583209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:11.583395 (+ 186us) server_negotiation.cc:407] Connection header received
0807 08:30:11.583521 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:11.583523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:11.583570 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:11.583625 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:11.584015 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.584538 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.585280 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:11.585502 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:11.585574 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:11.587885 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:48969: BlockingRecv error: recv got EOF from 127.1.157.67:48969 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:12.232621 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:12.227695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:54245)
0807 08:30:12.227940 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:12.227955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:12.228041 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:12.228263 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:12.228265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:12.228440 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:12.228538 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.228543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.229225 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.229227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.229769 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.229775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.230214 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.230217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.230270 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.230784 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:12.230797 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:12.232504 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":70,"threads_started":1}
W20260807 08:30:12.232776 15286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:12.227784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:54245 (local address 127.1.157.126:46713)
0807 08:30:12.228007 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:12.228010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:12.228023 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:12.228114 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:12.228117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:12.228162 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:12.228240 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:12.228617 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.229151 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.229886 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.230127 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.230214 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.232643 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:54245: BlockingRecv error: recv got EOF from 127.1.157.65:54245 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":67,"threads_started":1}
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:12.415155 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:12.410117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:58869)
0807 08:30:12.410404 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:12.410420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:12.410541 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:12.410838 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:12.410841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:12.411070 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:12.411177 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.411183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.411886 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.411888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.412427 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.412433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.412833 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.412835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.412891 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.413343 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:12.413354 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:12.415044 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
W20260807 08:30:12.415302 15286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:12.410164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58869 (local address 127.1.157.126:46713)
0807 08:30:12.410322 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:12.410327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:12.410488 (+ 161us) server_negotiation.cc:407] Connection header received
0807 08:30:12.410636 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:12.410639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:12.410690 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:12.410754 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:12.411289 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.411808 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.412528 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.412747 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.412826 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.415173 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:58869: BlockingRecv error: recv got EOF from 127.1.157.66:58869 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:12 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:12.593685 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:12.588597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:40475)
0807 08:30:12.588888 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:12.588901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:12.588998 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:12.589279 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:12.589282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:12.589508 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:12.589606 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.589612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.590377 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.590381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.590926 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:12.590934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.591365 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.591369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:12.591450 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.591930 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:12.591943 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:12.593542 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
W20260807 08:30:12.593812 15286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:12.588707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:40475 (local address 127.1.157.126:46713)
0807 08:30:12.588849 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:12.588853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:12.588954 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:30:12.589116 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:12.589119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:12.589168 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:12.589232 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:12.589716 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.590243 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.591041 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:12.591263 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:12.591351 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:12.593683 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:40475: BlockingRecv error: recv got EOF from 127.1.157.67:40475 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:13.238601 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:13.233402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:33985)
0807 08:30:13.233684 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:13.233697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:13.233796 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:13.234006 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:13.234008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:13.234189 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:13.234287 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.234292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.235000 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.235002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.235560 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.235567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.235993 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.235996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.236059 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.236569 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:13.236591 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:13.238452 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
W20260807 08:30:13.238696 15292 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:13.233447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:33985 (local address 127.1.157.126:46713)
0807 08:30:13.233709 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:13.233712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:13.233749 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:30:13.233868 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:13.233870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:13.233910 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:13.233982 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:13.234396 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.234922 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.235679 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.235906 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.235980 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.238585 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:33985: BlockingRecv error: recv got EOF from 127.1.157.65:33985 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:13.421116 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:13.415917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:36839)
0807 08:30:13.416194 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:13.416210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:13.416334 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:13.416579 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:13.416581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:13.416792 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:13.416887 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.416892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.417619 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.417623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.418174 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.418182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.418663 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.418667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.418753 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.419206 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:13.419216 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:13.420944 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
W20260807 08:30:13.421200 15292 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:13.416078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:36839 (local address 127.1.157.126:46713)
0807 08:30:13.416217 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:13.416222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:13.416280 (+ 58us) server_negotiation.cc:407] Connection header received
0807 08:30:13.416421 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:13.416424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:13.416473 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:13.416540 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:13.417001 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.417516 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.418287 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.418551 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.418636 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.421099 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:36839: BlockingRecv error: recv got EOF from 127.1.157.66:36839 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:13 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:13.600087 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:13.594528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:46965)
0807 08:30:13.594792 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:13.594807 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:13.594915 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:13.595174 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:13.595177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:13.595381 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:13.595473 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.595479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.596216 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.596219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.596786 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:13.596795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.597223 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.597227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:13.597308 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.597801 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:13.597815 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:13.599934 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
W20260807 08:30:13.600172 15292 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:13.594626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:46965 (local address 127.1.157.126:46713)
0807 08:30:13.594776 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:13.594781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:13.594864 (+ 83us) server_negotiation.cc:407] Connection header received
0807 08:30:13.595003 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:13.595006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:13.595054 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:13.595119 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:13.595587 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.596112 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.596900 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:13.597122 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:13.597209 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:13.600071 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:46965: BlockingRecv error: recv got EOF from 127.1.157.67:46965 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:14.244117 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:14.239332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:34541)
0807 08:30:14.239590 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:14.239603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:14.239702 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:14.239924 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:14.239926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:14.240103 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:14.240200 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.240205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.240885 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.240887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.241447 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.241454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.241816 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.241818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.241878 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.242335 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:14.242371 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:14.243999 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
W20260807 08:30:14.244277 15299 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:14.239375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:34541 (local address 127.1.157.126:46713)
0807 08:30:14.239597 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:14.239601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:14.239666 (+ 65us) server_negotiation.cc:407] Connection header received
0807 08:30:14.239769 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:14.239771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:14.239820 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:14.239902 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:14.240284 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.240805 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.241528 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.241740 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.241806 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.244142 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:34541: BlockingRecv error: recv got EOF from 127.1.157.65:34541 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:14.427002 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:14.421886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:47957)
0807 08:30:14.422159 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:14.422173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:14.422278 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:14.422529 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:14.422532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:14.422732 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:14.422839 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.422845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.423598 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.423601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.424159 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.424167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.424605 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.424608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.424690 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.425143 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:14.425154 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:14.426861 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
W20260807 08:30:14.427120 15299 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:14.421969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47957 (local address 127.1.157.126:46713)
0807 08:30:14.422111 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:14.422115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:14.422240 (+ 125us) server_negotiation.cc:407] Connection header received
0807 08:30:14.422377 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:14.422380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:14.422427 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:14.422496 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:14.422951 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.423492 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.424278 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.424504 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.424589 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.426994 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:47957: BlockingRecv error: recv got EOF from 127.1.157.66:47957 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:14 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:14.606611 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:14.600922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:51423)
0807 08:30:14.601235 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:14.601250 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:14.601364 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:14.601597 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:14.601599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:14.601812 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:14.601906 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.601912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.602681 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.602684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.603229 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:14.603238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.603851 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.603854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:14.603943 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.604454 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:14.604468 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:14.606455 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":88,"threads_started":1}
W20260807 08:30:14.606909 15299 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:14.601036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:51423 (local address 127.1.157.126:46713)
0807 08:30:14.601215 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:14.601220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:14.601321 (+ 101us) server_negotiation.cc:407] Connection header received
0807 08:30:14.601450 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:14.601453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:14.601497 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:14.601561 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:14.602013 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.602582 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.603378 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:14.603742 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:14.603855 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:14.606622 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:51423: BlockingRecv error: recv got EOF from 127.1.157.67:51423 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:15.250202 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:15.244853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:34391)
0807 08:30:15.245161 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:15.245175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:15.245272 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:15.245482 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:15.245484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:15.245666 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:15.245765 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.245771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.246654 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.246657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.247313 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.247326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.247781 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.247785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.247871 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.248318 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:15.248329 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:15.250025 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":70,"threads_started":1}
W20260807 08:30:15.250288 15304 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:15.244924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:34391 (local address 127.1.157.126:46713)
0807 08:30:15.245155 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:15.245158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:15.245235 (+ 77us) server_negotiation.cc:407] Connection header received
0807 08:30:15.245339 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:15.245341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:15.245381 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:15.245455 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:15.245854 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.246521 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.247433 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.247669 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.247756 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.250172 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:34391: BlockingRecv error: recv got EOF from 127.1.157.65:34391 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:15.432706 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:15.427715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:59297)
0807 08:30:15.428009 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:15.428023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:15.428117 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:15.428320 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:15.428322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:15.428497 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:15.428593 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.428598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.429266 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.429268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.429810 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.429817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.430204 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.430207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.430268 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.430777 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:15.430788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:15.432556 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":70,"threads_started":1}
W20260807 08:30:15.432777 15304 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:15.427800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:59297 (local address 127.1.157.126:46713)
0807 08:30:15.427925 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:15.427929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:15.428081 (+ 152us) server_negotiation.cc:407] Connection header received
0807 08:30:15.428186 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:15.428188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:15.428230 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:15.428295 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:15.428680 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.429186 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.429898 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.430112 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.430186 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.432679 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:59297: BlockingRecv error: recv got EOF from 127.1.157.66:59297 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:15 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:15.616395 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:15.607318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:36385)
0807 08:30:15.607613 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:15.607626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:15.607726 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:15.607929 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:15.607931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:15.608111 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:15.608215 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.608220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.608878 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.608880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.609433 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:15.609440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.609794 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.609796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:15.609855 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.614188 (+ 4333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:15.614201 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:15.616249 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":71,"threads_started":1}
W20260807 08:30:15.616569 15304 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:15.607383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:36385 (local address 127.1.157.126:46713)
0807 08:30:15.607498 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:15.607501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:15.607679 (+ 178us) server_negotiation.cc:407] Connection header received
0807 08:30:15.607800 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:15.607802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:15.607846 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:15.607902 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:15.608299 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.608804 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.609513 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:15.609717 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:15.609781 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:15.616378 (+ 6597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:36385: BlockingRecv error: recv got EOF from 127.1.157.67:36385 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:16.256182 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:16.250912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:36387)
0807 08:30:16.251172 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:16.251186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:16.251288 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:16.251496 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:16.251498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:16.251671 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:16.251768 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.251773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.252451 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.252452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.253088 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.253101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.253712 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.253716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.253784 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.254253 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:16.254265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:16.256067 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
W20260807 08:30:16.256347 15309 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:16.251029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:36387 (local address 127.1.157.126:46713)
0807 08:30:16.251264 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:16.251268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:16.251281 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:16.251361 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:16.251363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:16.251403 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:16.251474 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:16.251846 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.252375 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.253194 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.253499 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.253602 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.256215 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:36387: BlockingRecv error: recv got EOF from 127.1.157.65:36387 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:16.438798 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:16.433406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:40187)
0807 08:30:16.433668 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:16.433683 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:16.433789 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:16.433976 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:16.433978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:16.434153 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:16.434259 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.434264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.435193 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.435195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.435818 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.435829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.436304 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.436307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.436380 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.436919 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:16.436932 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:16.438647 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":83,"threads_started":1}
W20260807 08:30:16.438889 15309 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:16.433466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40187 (local address 127.1.157.126:46713)
0807 08:30:16.433603 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:16.433607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:16.433752 (+ 145us) server_negotiation.cc:407] Connection header received
0807 08:30:16.433853 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:16.433855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:16.433900 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:16.433963 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:16.434359 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.435095 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.435904 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.436161 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.436250 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.438788 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:40187: BlockingRecv error: recv got EOF from 127.1.157.66:40187 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:16 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:16.621974 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:16.617073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:42395)
0807 08:30:16.617338 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:16.617351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:16.617448 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:16.617657 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:16.617659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:16.617827 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:16.617926 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.617931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.618661 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.618663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.619209 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:16.619217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.619591 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.619594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:16.619666 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.620130 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:16.620142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:16.621862 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":106,"threads_started":1}
W20260807 08:30:16.622092 15309 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:16.617139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:42395 (local address 127.1.157.126:46713)
0807 08:30:16.617269 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:16.617274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:16.617409 (+ 135us) server_negotiation.cc:407] Connection header received
0807 08:30:16.617516 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:16.617518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:16.617564 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:16.617625 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:16.618008 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.618570 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.619299 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:16.619511 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:16.619583 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:16.621988 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:42395: BlockingRecv error: recv got EOF from 127.1.157.67:42395 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:17.262105 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:17.256842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:39259)
0807 08:30:17.257107 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:17.257120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:17.257220 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:17.257438 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:17.257440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:17.257613 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:17.257712 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.257717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.258430 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.258432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.258982 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.258989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.259526 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.259530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.259598 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.260146 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:17.260158 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:17.261978 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
W20260807 08:30:17.262280 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:17.256934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39259 (local address 127.1.157.126:46713)
0807 08:30:17.257162 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:17.257166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:17.257178 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:30:17.257297 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:17.257299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:17.257340 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:17.257412 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:17.257795 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.258328 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.259089 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.259412 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.259489 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.262148 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:39259: BlockingRecv error: recv got EOF from 127.1.157.65:39259 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":59,"threads_started":1}
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:17.444774 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:17.439697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:57655)
0807 08:30:17.439967 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:17.439981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:17.440081 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:17.440285 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:17.440287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:17.440464 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:17.440565 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.440570 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.441273 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.441276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.441834 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.441843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.442196 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.442199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.442257 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.442768 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:17.442780 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:17.444612 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":108,"threads_started":1}
W20260807 08:30:17.444862 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:17.439741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57655 (local address 127.1.157.126:46713)
0807 08:30:17.439871 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:17.439874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:17.440040 (+ 166us) server_negotiation.cc:407] Connection header received
0807 08:30:17.440149 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:17.440151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:17.440193 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:17.440255 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:17.440650 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.441186 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.441921 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.442123 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.442189 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.444761 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:57655: BlockingRecv error: recv got EOF from 127.1.157.66:57655 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:17 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:17.627548 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:17.622636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:49965)
0807 08:30:17.622883 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:17.622897 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:17.622999 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:17.623247 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:17.623250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:17.623428 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:17.623527 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.623532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.624215 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.624217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.624750 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:17.624756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.625105 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.625107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:17.625175 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.625609 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:17.625621 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:17.627425 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
W20260807 08:30:17.627669 15315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:17.622702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:49965 (local address 127.1.157.126:46713)
0807 08:30:17.622834 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:17.622839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:17.622961 (+ 122us) server_negotiation.cc:407] Connection header received
0807 08:30:17.623065 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:17.623067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:17.623113 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:17.623179 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:17.623627 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.624140 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.624825 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:17.625032 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:17.625096 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:17.627546 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:49965: BlockingRecv error: recv got EOF from 127.1.157.67:49965 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:18.268141 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:18.262856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:39855)
0807 08:30:18.263126 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:18.263140 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:18.263231 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:18.263428 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:18.263430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:18.263605 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:18.263713 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.263719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.264413 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.264415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.264972 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.264980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.265359 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.265361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.265419 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.265944 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:18.265957 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:18.267990 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
W20260807 08:30:18.268355 15322 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:18.262941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39855 (local address 127.1.157.126:46713)
0807 08:30:18.263210 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:18.263214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:18.263227 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:18.263300 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:18.263302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:18.263344 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:18.263414 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:18.263797 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.264335 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.265057 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.265281 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.265348 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.268227 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:39855: BlockingRecv error: recv got EOF from 127.1.157.65:39855 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:18.451068 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:18.445551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:33783)
0807 08:30:18.445864 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:18.445879 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:18.445997 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:18.446281 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:18.446284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:18.446504 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:18.446606 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.446612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.447414 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.447417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.447970 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.447979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.448415 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.448419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.448502 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.448970 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:18.448981 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:18.450955 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":89,"threads_started":1}
W20260807 08:30:18.451201 15322 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:18.445625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:33783 (local address 127.1.157.126:46713)
0807 08:30:18.445804 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:18.445809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:18.445951 (+ 142us) server_negotiation.cc:407] Connection header received
0807 08:30:18.446096 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:18.446100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:18.446171 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:18.446236 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:18.446735 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.447309 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.448085 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.448314 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.448400 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.451078 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:33783: BlockingRecv error: recv got EOF from 127.1.157.66:33783 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:18 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:18.633438 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:18.628216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:40119)
0807 08:30:18.628487 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:18.628499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:18.628592 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:18.628817 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:18.628820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:18.629014 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:18.629114 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.629120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.629993 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.629997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.630755 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:18.630765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.631168 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.631170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:18.631223 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.631663 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:18.631674 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:18.633302 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
W20260807 08:30:18.633661 15322 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:18.628334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:40119 (local address 127.1.157.126:46713)
0807 08:30:18.628476 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:18.628481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:18.628549 (+ 68us) server_negotiation.cc:407] Connection header received
0807 08:30:18.628660 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:18.628663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:18.628707 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:18.628777 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:18.629222 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.629872 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.630855 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:18.631083 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:18.631161 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:18.633446 (+ 2285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:40119: BlockingRecv error: recv got EOF from 127.1.157.67:40119 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:19.273998 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:19.268895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:40575)
0807 08:30:19.269153 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:19.269166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:19.269264 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:19.269465 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:19.269467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:19.269644 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:19.269743 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.269748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.270497 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.270501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.271076 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.271087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.271487 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.271490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.271546 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.272063 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:19.272074 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:19.273832 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
W20260807 08:30:19.274085 15329 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:19.268958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40575 (local address 127.1.157.126:46713)
0807 08:30:19.269209 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:19.269212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:19.269226 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:30:19.269339 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:19.269341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:19.269383 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:19.269453 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:19.269824 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.270385 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.271175 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.271404 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.271476 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.273970 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:40575: BlockingRecv error: recv got EOF from 127.1.157.65:40575 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":59,"threads_started":1}
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:19.456997 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:19.451803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:40693)
0807 08:30:19.452070 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:19.452083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:19.452186 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:19.452395 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:19.452397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:19.452576 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:19.452683 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.452688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.453359 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.453361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.453929 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.453936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.454291 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.454293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.454382 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.454862 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:19.454874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:19.456860 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":72,"threads_started":1}
W20260807 08:30:19.457078 15329 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:19.451877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:40693 (local address 127.1.157.126:46713)
0807 08:30:19.452019 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:19.452024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:19.452147 (+ 123us) server_negotiation.cc:407] Connection header received
0807 08:30:19.452258 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:19.452260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:19.452304 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:19.452369 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:19.452765 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.453281 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.454007 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.454215 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.454282 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.456976 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:40693: BlockingRecv error: recv got EOF from 127.1.157.66:40693 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:19 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:19.639314 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:19.634152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:53557)
0807 08:30:19.634443 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:19.634454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:19.634565 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:19.634809 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:19.634812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:19.635012 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:19.635109 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.635115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.635922 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.635926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.636535 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:19.636570 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.636994 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.636998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:19.637089 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.637521 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:19.637532 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:19.639169 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":78,"threads_started":1}
W20260807 08:30:19.639411 15329 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:19.634213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:53557 (local address 127.1.157.126:46713)
0807 08:30:19.634365 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:19.634369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:19.634520 (+ 151us) server_negotiation.cc:407] Connection header received
0807 08:30:19.634659 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:19.634662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:19.634711 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:19.634777 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:19.635239 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.635795 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.636671 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:19.636895 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:19.636981 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:19.639312 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:53557: BlockingRecv error: recv got EOF from 127.1.157.67:53557 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:20.279667 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:20.274679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:57201)
0807 08:30:20.274947 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:20.274960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:20.275047 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:20.275270 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:20.275272 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:20.275446 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:20.275545 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.275551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.276239 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.276240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.276774 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.276781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.277136 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.277138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.277207 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.277699 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:20.277710 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:20.279504 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
W20260807 08:30:20.279759 15341 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:20.274745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:57201 (local address 127.1.157.126:46713)
0807 08:30:20.274979 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:20.274982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:20.274998 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:30:20.275114 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:20.275116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:20.275169 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:20.275243 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:20.275631 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.276157 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.276850 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.277059 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.277140 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.279644 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:57201: BlockingRecv error: recv got EOF from 127.1.157.65:57201 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":60,"threads_started":1}
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:20.462481 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:20.457660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:48511)
0807 08:30:20.457910 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:20.457923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:20.458021 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:20.458235 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:20.458238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:20.458440 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:20.458541 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.458547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.459211 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.459213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.459763 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.459770 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.460121 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.460123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.460177 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.460624 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:20.460643 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:20.462321 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
W20260807 08:30:20.462572 15341 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:20.457735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:48511 (local address 127.1.157.126:46713)
0807 08:30:20.457862 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:20.457866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:20.457985 (+ 119us) server_negotiation.cc:407] Connection header received
0807 08:30:20.458090 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:20.458092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:20.458134 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:20.458211 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:20.458625 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.459131 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.459843 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.460047 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.460112 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.462467 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:48511: BlockingRecv error: recv got EOF from 127.1.157.66:48511 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:20 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:20.653388 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:20.640407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:42907)
0807 08:30:20.640819 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:20.640833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:20.640990 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:20.641481 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:20.641494 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:20.641827 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:20.642064 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.642074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.643264 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.643268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.644093 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:20.644110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.644624 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.644627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:20.644725 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.650226 (+ 5501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:20.650241 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:20.653093 (+ 2852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":107,"threads_started":1}
W20260807 08:30:20.653606 15341 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:20.640481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:42907 (local address 127.1.157.126:46713)
0807 08:30:20.640715 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:20.640730 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:20.640928 (+ 198us) server_negotiation.cc:407] Connection header received
0807 08:30:20.641149 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:20.641153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:20.641286 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:20.641459 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:20.642252 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.643101 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.644264 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:20.644518 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:20.644601 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:20.653374 (+ 8773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:42907: BlockingRecv error: recv got EOF from 127.1.157.67:42907 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:21.289337 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:21.280532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:51123)
0807 08:30:21.281063 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:21.281097 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:21.281276 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:21.281748 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:21.281762 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:21.282095 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:21.282289 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.282305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.283566 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.283571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.284559 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.284578 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.285332 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.285337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.285496 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.286259 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:21.286282 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:21.289054 (+ 2772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":223,"threads_started":1}
W20260807 08:30:21.289529 15346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:21.280600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:51123 (local address 127.1.157.126:46713)
0807 08:30:21.281064 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:21.281087 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:21.281187 (+ 100us) server_negotiation.cc:407] Connection header received
0807 08:30:21.281454 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:21.281457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:21.281549 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:21.281685 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:21.282515 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.283398 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.284746 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.285152 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.285334 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.289312 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:51123: BlockingRecv error: recv got EOF from 127.1.157.65:51123 (error 108)
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":168,"threads_started":1}
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:21.476751 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:21.463725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:56695)
0807 08:30:21.464383 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:21.464418 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:21.464804 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:21.465435 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:21.465448 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:21.466000 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:21.466267 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.466304 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.467836 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.467854 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.469351 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.469369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.470311 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.470314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.470522 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.471732 (+ 1210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:21.471770 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:21.476171 (+ 4401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":223,"threads_started":1}
W20260807 08:30:21.476958 15346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:21.464012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:56695 (local address 127.1.157.126:46713)
0807 08:30:21.464319 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:21.464325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:21.464581 (+ 256us) server_negotiation.cc:407] Connection header received
0807 08:30:21.465031 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:21.465042 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:21.465200 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:21.465354 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:21.466508 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.467652 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.469561 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.470165 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.470280 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.476536 (+ 6256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:56695: BlockingRecv error: recv got EOF from 127.1.157.66:56695 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:21 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:21.662647 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:21.654444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:41985)
0807 08:30:21.654801 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:21.654823 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:21.654989 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:21.655345 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:21.655347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:21.655609 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:21.655837 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.655845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.656900 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.656902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.657831 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:21.657849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.658651 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.658657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:21.658815 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.659719 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:21.659758 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:21.662476 (+ 2718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":119,"threads_started":1}
W20260807 08:30:21.662845 15346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:21.654479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:41985 (local address 127.1.157.126:46713)
0807 08:30:21.654687 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:21.654692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:21.654899 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:30:21.655088 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:21.655090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:21.655210 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:21.655311 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:21.655995 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.656762 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.658018 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:21.658462 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:21.658632 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:21.662650 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:41985: BlockingRecv error: recv got EOF from 127.1.157.67:41985 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:22.295796 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:22.290408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:51953)
0807 08:30:22.290807 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:22.290823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:22.290937 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:22.291222 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:22.291225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:22.291464 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:22.291588 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.291595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.292366 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.292368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.292953 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.292960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.293357 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.293359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.293418 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.293899 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:22.293910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:22.295664 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":89,"threads_started":1}
W20260807 08:30:22.295920 15352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:22.290462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:51953 (local address 127.1.157.126:46713)
0807 08:30:22.290793 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:22.290796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:22.290898 (+ 102us) server_negotiation.cc:407] Connection header received
0807 08:30:22.291019 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:22.291022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:22.291073 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:22.291185 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:22.291707 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.292279 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.293039 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.293276 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.293347 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.295807 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:51953: BlockingRecv error: recv got EOF from 127.1.157.65:51953 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":79,"threads_started":1}
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:22.483006 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:22.477735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:49493)
0807 08:30:22.477975 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:22.477990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:22.478086 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:22.478297 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:22.478299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:22.478501 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:22.478607 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.478614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.479317 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.479319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.479864 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.479871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.480296 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.480298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.480356 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.480787 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:22.480798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:22.482841 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
W20260807 08:30:22.483076 15352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:22.477795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:49493 (local address 127.1.157.126:46713)
0807 08:30:22.477938 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:22.477942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:22.478050 (+ 108us) server_negotiation.cc:407] Connection header received
0807 08:30:22.478161 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:22.478162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:22.478206 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:22.478272 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:22.478702 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.479234 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.479967 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.480211 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.480288 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.482975 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:49493: BlockingRecv error: recv got EOF from 127.1.157.66:49493 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:22 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.67@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:22.668494 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:22.663392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.67:45337)
0807 08:30:22.663668 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:22.663682 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:22.663794 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:22.664030 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:22.664033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:22.664249 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:22.664346 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.664353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.665089 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.665093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.665638 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:22.665648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.666084 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.666087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:22.666184 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.666648 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:22.666659 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:22.668366 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":72,"threads_started":1}
W20260807 08:30:22.668615 15352 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:22.663462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:45337 (local address 127.1.157.126:46713)
0807 08:30:22.663604 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:22.663608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:22.663751 (+ 143us) server_negotiation.cc:407] Connection header received
0807 08:30:22.663878 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:22.663881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:22.663927 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:22.663993 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:22.664463 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.664985 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.665753 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:22.665977 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:22.666062 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:22.668501 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.67:45337: BlockingRecv error: recv got EOF from 127.1.157.67:45337 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 08:30:23 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:23 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.65@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:23.301875 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:23.296720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.65:49537)
0807 08:30:23.296973 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:23.296985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:23.297070 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:23.297343 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:23.297345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:23.297527 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:23.297620 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:23.297625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.298301 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.298303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:23.298860 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:23.298867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.299216 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.299219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:23.299275 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:23.299777 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:23.299791 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:23.301644 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
W20260807 08:30:23.301919 15356 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:23.296810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:49537 (local address 127.1.157.126:46713)
0807 08:30:23.297022 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:23.297028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:23.297041 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:23.297195 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:23.297198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:23.297248 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:23.297331 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:23.297701 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.298226 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.298933 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.299142 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.299209 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:23.301811 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.65:49537: BlockingRecv error: recv got EOF from 127.1.157.65:49537 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":57,"threads_started":1}
Aug 07 08:30:23 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 08:30:23 dist-test-slave-6zbq krb5kdc[13697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.157.66@KRBTEST.COM for oryx/127.1.157.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 08:30:23.488739 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:23.483917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46713 (local address 127.1.157.66:47421)
0807 08:30:23.484193 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:23.484209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:23.484303 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:23.484501 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:23.484503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:23.484678 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:23.484781 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:23.484786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.485457 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.485459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:23.486001 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:23.486008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.486394 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.486396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:23.486456 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:23.486891 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:23.486902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:23.488567 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.157.126:46713: Server oryx/127.1.157.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
W20260807 08:30:23.488808 15356 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:23.484008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47421 (local address 127.1.157.126:46713)
0807 08:30:23.484159 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:23.484163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:23.484259 (+ 96us) server_negotiation.cc:407] Connection header received
0807 08:30:23.484378 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:23.484380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:23.484419 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:23.484472 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:23.484861 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.485384 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.486079 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.486291 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.486384 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:23.488708 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.157.66:47421: BlockingRecv error: recv got EOF from 127.1.157.66:47421 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:30:23.559108 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 14613
I20260807 08:30:23.570250 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 14852
I20260807 08:30:23.582273 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 15053
I20260807 08:30:23.589162 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 14492
2026-08-07T08:30:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18014 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T08:30:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:23Z Disabled control of system clock
I20260807 08:30:23.624157 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42287
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 08:30:23.728895 15363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:23.729113 15363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:23.729146 15363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:23.733012 15363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:23.733062 15363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:23.733075 15363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:23.733090 15363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:23.733103 15363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:23.737630 15363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:23.738567 15363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:23.739768 15363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:23.747628 15377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:23.747622 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:23.747637 15371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:23.747730 15363 server_base.cc:1034] running on GCE node
I20260807 08:30:23.748390 15363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:23.749331 15363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:23.750466 15363 hybrid_clock.cc:648] HybridClock initialized: now 1786091423750450 us; error 27 us; skew 500 ppm
I20260807 08:30:23.752128 15363 webserver.cc:460] Webserver started at http://127.1.157.126:44143/ using document root <none> and password file <none>
I20260807 08:30:23.752629 15363 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:23.752692 15363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:23.752882 15363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:23.754776 15363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "da82c01d3c764d84b587748c5f136603"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f4fd2f773d09934827427207c700e611"
server_key_iv: "a8c3083b3238fd0cc2fcb58c6053df82"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.755287 15363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "da82c01d3c764d84b587748c5f136603"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f4fd2f773d09934827427207c700e611"
server_key_iv: "a8c3083b3238fd0cc2fcb58c6053df82"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.758625 15363 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:23.760797 15385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:23.761904 15363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:23.762010 15363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "da82c01d3c764d84b587748c5f136603"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f4fd2f773d09934827427207c700e611"
server_key_iv: "a8c3083b3238fd0cc2fcb58c6053df82"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.762099 15363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:23.771360 15363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:23.771893 15363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:23.772039 15363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:23.778100 15363 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:42287
I20260807 08:30:23.778108 15458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:42287 every 8 connection(s)
I20260807 08:30:23.778930 15363 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:23.781467 15459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:23.786481 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: Bootstrap starting.
I20260807 08:30:23.788671 15459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:23.788992 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 15363
I20260807 08:30:23.789083 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:30:23.789333 1653 external_mini_cluster.cc:1428] Setting key ded7055d1723b9620d68582ded2acc3b
I20260807 08:30:23.789414 15459 log.cc:824] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:23.791338 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: No bootstrap required, opened a new log
I20260807 08:30:23.795028 15459 raft_consensus.cc:348] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } }
I20260807 08:30:23.795282 15459 raft_consensus.cc:374] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:23.795344 15459 raft_consensus.cc:729] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da82c01d3c764d84b587748c5f136603, State: Initialized, Role: FOLLOWER
I20260807 08:30:23.795778 15459 consensus_queue.cc:260] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } }
I20260807 08:30:23.795931 15459 raft_consensus.cc:388] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:23.795997 15459 raft_consensus.cc:482] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:23.796087 15459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:23.796996 15459 raft_consensus.cc:504] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } }
I20260807 08:30:23.797297 15459 leader_election.cc:302] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da82c01d3c764d84b587748c5f136603; no voters:
I20260807 08:30:23.797538 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:23.790290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.1.157.126:42287)
0807 08:30:23.790862 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:23.790877 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:23.790909 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:30:23.791510 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:23.791526 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:23.791817 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:23.792061 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:23.792438 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.793328 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.794531 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:23.794960 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:23.795065 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:23.795867 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:23.795872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:23.795883 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:23.796314 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:23.796366 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:23.796385 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:23.796456 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:23.796603 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:30:23.796828 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":102,"threads_started":1}
I20260807 08:30:23.797552 15459 leader_election.cc:288] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:30:23.797636 15467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:23.797806 15467 raft_consensus.cc:686] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [term 1 LEADER]: Becoming Leader. State: Replica: da82c01d3c764d84b587748c5f136603, State: Running, Role: LEADER
I20260807 08:30:23.798079 15467 consensus_queue.cc:237] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } }
I20260807 08:30:23.798761 15459 sys_catalog.cc:564] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:23.799233 15470 sys_catalog.cc:455] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da82c01d3c764d84b587748c5f136603. Latest consensus state: current_term: 1 leader_uuid: "da82c01d3c764d84b587748c5f136603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } } }
I20260807 08:30:23.799265 15469 sys_catalog.cc:455] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da82c01d3c764d84b587748c5f136603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da82c01d3c764d84b587748c5f136603" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42287 } } }
I20260807 08:30:23.799350 15470 sys_catalog.cc:458] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:23.799362 15469 sys_catalog.cc:458] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603 [sys.catalog]: This master's current role is: LEADER
W20260807 08:30:23.803539 15492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:30:23.803615 15492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:30:23.803679 15475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:23.804461 15475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:23.813711 15475 catalog_manager.cc:1347] Generated new cluster ID: f579414a45264e9198ba93c71c68336e
I20260807 08:30:23.813769 15475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:23.820714 15475 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:30:23.821556 15475 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:23.827384 15475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da82c01d3c764d84b587748c5f136603: Generated new TSK 0
I20260807 08:30:23.827920 15475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:30:23.832633 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--builtin_ntp_servers=127.1.157.84:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 08:30:23.958954 15497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:23.959501 15497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:23.959633 15497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:23.964852 15497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:23.964951 15497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:23.965030 15497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:23.970469 15497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:23.971755 15497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:23.973178 15497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:23.983029 15508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:23.983080 15497 server_base.cc:1034] running on GCE node
W20260807 08:30:23.983029 15510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:23.983058 15507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:23.983668 15497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:23.984295 15497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:23.985448 15497 hybrid_clock.cc:648] HybridClock initialized: now 1786091423985423 us; error 40 us; skew 500 ppm
I20260807 08:30:23.987262 15497 webserver.cc:460] Webserver started at http://127.1.157.65:33269/ using document root <none> and password file <none>
I20260807 08:30:23.987818 15497 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:23.987885 15497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:23.988075 15497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:23.990097 15497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "5020067db12d40aca376b9d33b4365f2"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f05cf0cd77e09a65cbdff7961d2e5a99"
server_key_iv: "048a2a6d0cb8103d22a7cacf7988cb4e"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.990669 15497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "5020067db12d40aca376b9d33b4365f2"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f05cf0cd77e09a65cbdff7961d2e5a99"
server_key_iv: "048a2a6d0cb8103d22a7cacf7988cb4e"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.994122 15497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:23.996563 15518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:23.997678 15497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:23.997788 15497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "5020067db12d40aca376b9d33b4365f2"
format_stamp: "Formatted at 2026-08-07 08:30:23 on dist-test-slave-6zbq"
server_key: "f05cf0cd77e09a65cbdff7961d2e5a99"
server_key_iv: "048a2a6d0cb8103d22a7cacf7988cb4e"
server_key_version: "encryptionkey@0"
I20260807 08:30:23.997879 15497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:24.023825 15497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:24.024437 15497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:24.024597 15497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:24.025130 15497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:24.026077 15497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:24.026127 15497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.026178 15497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:24.026214 15497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.034132 15497 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:38819
I20260807 08:30:24.034165 15683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:38819 every 8 connection(s)
I20260807 08:30:24.034940 15497 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:24.040208 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 15497
I20260807 08:30:24.040323 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:30:24.040596 1653 external_mini_cluster.cc:1428] Setting key da76dae75dcab04fe1f5ddbc370470b3
I20260807 08:30:24.042594 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--builtin_ntp_servers=127.1.157.84:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:30:24.043839 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.036553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:42585 (local address 127.1.157.126:42287)
0807 08:30:24.036697 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.036702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.037669 (+ 967us) server_negotiation.cc:407] Connection header received
0807 08:30:24.039005 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.039008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.039059 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.039123 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.039886 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.040631 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.042032 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.042360 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.042467 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.043409 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:24.043412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:24.043415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:24.043428 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:24.043451 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:24.043461 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:24.043533 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:24.043634 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 08:30:24.043701 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:30:24.044564 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.036930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42287 (local address 127.1.157.65:42585)
0807 08:30:24.037534 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:24.037571 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:24.038819 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:24.039257 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:24.039264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:24.039340 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:24.039784 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.039796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.040754 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.040757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.041910 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.041921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.042489 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.042492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.042623 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.043169 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:24.043170 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:24.043207 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:24.043266 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:24.043531 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:24.043535 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:24.043539 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:30:24.043617 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:30:24.043821 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260807 08:30:24.045629 15686 heartbeater.cc:347] Connected to a master server at 127.1.157.126:42287
I20260807 08:30:24.045828 15686 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:24.046270 15686 heartbeater.cc:511] Master 127.1.157.126:42287 requested a full tablet report, sending...
I20260807 08:30:24.047804 15413 ts_manager.cc:194] Registered new tserver with Master: 5020067db12d40aca376b9d33b4365f2 (127.1.157.65:38819)
I20260807 08:30:24.049340 15413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.65:42585
W20260807 08:30:24.148936 15689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:24.149140 15689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:24.149173 15689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:24.153009 15689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:24.153051 15689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:24.153100 15689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:24.157465 15689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:24.158291 15689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:24.159453 15689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:24.167858 15695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.167909 15698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.168001 15689 server_base.cc:1034] running on GCE node
W20260807 08:30:24.167922 15700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.168495 15689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:24.169034 15689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:24.170192 15689 hybrid_clock.cc:648] HybridClock initialized: now 1786091424170172 us; error 30 us; skew 500 ppm
I20260807 08:30:24.171792 15689 webserver.cc:460] Webserver started at http://127.1.157.66:45059/ using document root <none> and password file <none>
I20260807 08:30:24.172273 15689 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:24.172338 15689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:24.172509 15689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:24.174288 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "12cc32c84ea440ce806e203d22ac0c1c"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "a09012248257ce4783eaa860161a84ba"
server_key_iv: "f42c7828b5ef9c1c9215d602ba36e662"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.174856 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "12cc32c84ea440ce806e203d22ac0c1c"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "a09012248257ce4783eaa860161a84ba"
server_key_iv: "f42c7828b5ef9c1c9215d602ba36e662"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.178047 15689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:24.180310 15709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.181371 15689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:24.181473 15689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "12cc32c84ea440ce806e203d22ac0c1c"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "a09012248257ce4783eaa860161a84ba"
server_key_iv: "f42c7828b5ef9c1c9215d602ba36e662"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.181562 15689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:24.223091 15689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:24.223800 15689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:24.223969 15689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:24.224519 15689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:24.225407 15689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:24.225467 15689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.225519 15689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:24.225548 15689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.233615 15689 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:40501
I20260807 08:30:24.233649 15901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:40501 every 8 connection(s)
I20260807 08:30:24.234472 15689 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:24.239333 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 15689
I20260807 08:30:24.239449 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:30:24.239768 1653 external_mini_cluster.cc:1428] Setting key 8aba380ea87de46da9c0824a3c30ae90
I20260807 08:30:24.241714 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--builtin_ntp_servers=127.1.157.84:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 08:30:24.243418 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.236100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47753 (local address 127.1.157.126:42287)
0807 08:30:24.236230 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.236234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.237185 (+ 951us) server_negotiation.cc:407] Connection header received
0807 08:30:24.238532 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.238535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.238592 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.238670 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.239670 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.240265 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.241578 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.241820 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.241917 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.242965 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:24.242969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:24.242972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:24.242984 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:24.243004 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:24.243012 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:24.243071 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:24.243221 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 08:30:24.243282 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:30:24.244165 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.236444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42287 (local address 127.1.157.66:47753)
0807 08:30:24.237046 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:24.237084 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:24.238310 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:24.238840 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:24.238847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:24.238925 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:24.239510 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.239531 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.240395 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.240398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.241459 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.241472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.242045 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.242047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.242178 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.242728 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:24.242729 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:24.242766 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:24.242826 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:24.243080 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:24.243089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:24.243095 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:30:24.243197 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 08:30:24.243455 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260807 08:30:24.245136 15903 heartbeater.cc:347] Connected to a master server at 127.1.157.126:42287
I20260807 08:30:24.245358 15903 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:24.245936 15903 heartbeater.cc:511] Master 127.1.157.126:42287 requested a full tablet report, sending...
I20260807 08:30:24.247021 15413 ts_manager.cc:194] Registered new tserver with Master: 12cc32c84ea440ce806e203d22ac0c1c (127.1.157.66:40501)
I20260807 08:30:24.247758 15413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.66:47753
W20260807 08:30:24.349082 15909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:24.349295 15909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:24.349330 15909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:24.353281 15909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:24.353332 15909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:24.353380 15909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:24.357904 15909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:42287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:24.358843 15909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:24.360059 15909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:24.369029 15920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.369156 15909 server_base.cc:1034] running on GCE node
W20260807 08:30:24.369069 15916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.369037 15918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.369699 15909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:24.370232 15909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:24.371382 15909 hybrid_clock.cc:648] HybridClock initialized: now 1786091424371368 us; error 34 us; skew 500 ppm
I20260807 08:30:24.373087 15909 webserver.cc:460] Webserver started at http://127.1.157.67:39123/ using document root <none> and password file <none>
I20260807 08:30:24.373600 15909 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:24.373659 15909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:24.373844 15909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:24.375774 15909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "956fc90342d343efa4c40428ea3abe6d"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "f5d67de3d5248357945a4bc2ddb4bb00"
server_key_iv: "d1be7790ec02082ecbf4c4b45a2e15d8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.376291 15909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "956fc90342d343efa4c40428ea3abe6d"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "f5d67de3d5248357945a4bc2ddb4bb00"
server_key_iv: "d1be7790ec02082ecbf4c4b45a2e15d8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.379652 15909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:30:24.381848 15932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.382953 15909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:24.383056 15909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "956fc90342d343efa4c40428ea3abe6d"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "f5d67de3d5248357945a4bc2ddb4bb00"
server_key_iv: "d1be7790ec02082ecbf4c4b45a2e15d8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.383147 15909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:24.414589 15909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:24.415196 15909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:24.415359 15909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:24.415904 15909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:24.416785 15909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:24.416833 15909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.416883 15909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:24.416913 15909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.424780 15909 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:46509
I20260807 08:30:24.424808 16107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:46509 every 8 connection(s)
I20260807 08:30:24.425565 15909 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:30:24.428272 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 15909
I20260807 08:30:24.428390 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:30:24.428682 1653 external_mini_cluster.cc:1428] Setting key dffc57c9ff0ea97dbe7061e8f79e912a
I20260807 08:30:24.434173 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.427274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:40807 (local address 127.1.157.126:42287)
0807 08:30:24.427400 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.427404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.428452 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 08:30:24.429957 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.429960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.430014 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.430086 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.431124 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.431634 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.432661 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.432862 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.432926 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.433819 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:24.433822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:24.433824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:24.433835 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:24.433855 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:24.433862 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:24.433915 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:24.434018 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 08:30:24.434068 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:30:24.434938 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.427623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42287 (local address 127.1.157.67:40807)
0807 08:30:24.428289 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:24.428342 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:24.429761 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:24.430294 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:24.430303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:24.430439 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:24.431000 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.431014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.431731 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.431733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.432568 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:24.432576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:24.432963 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:24.432965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:24.433099 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:24.433595 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:24.433597 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:24.433632 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:24.433689 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:24.433921 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:24.433926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:24.433930 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:30:24.434010 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:30:24.434219 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260807 08:30:24.435875 16108 heartbeater.cc:347] Connected to a master server at 127.1.157.126:42287
I20260807 08:30:24.436070 16108 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:24.436507 16108 heartbeater.cc:511] Master 127.1.157.126:42287 requested a full tablet report, sending...
I20260807 08:30:24.437378 15413 ts_manager.cc:194] Registered new tserver with Master: 956fc90342d343efa4c40428ea3abe6d (127.1.157.67:46509)
I20260807 08:30:24.437906 15413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.67:40807
I20260807 08:30:24.442188 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:30:24.444698 15465 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:24.443972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.1.157.126:42287)
0807 08:30:24.444071 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.444091 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.444226 (+ 135us) server_negotiation.cc:407] Connection header received
0807 08:30:24.444315 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.444317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.444350 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.444414 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.444589 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43832: BlockingRecv error: recv got EOF from 127.0.0.1:43832 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:30:24.445029 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 15497
I20260807 08:30:24.460054 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 15689
I20260807 08:30:24.475984 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 15909
I20260807 08:30:24.482419 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 15363
2026-08-07T08:30:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (888 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T08:30:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:24Z Disabled control of system clock
I20260807 08:30:24.513598 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43125
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:30:24.621327 16127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:24.621536 16127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:24.621574 16127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:24.625442 16127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:24.625487 16127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:24.625501 16127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:24.625516 16127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:24.625545 16127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:24.630147 16127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:24.631059 16127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:24.632287 16127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:24.639887 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.640030 16127 server_base.cc:1034] running on GCE node
W20260807 08:30:24.640014 16136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.639896 16137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.640681 16127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:24.641669 16127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:24.642814 16127 hybrid_clock.cc:648] HybridClock initialized: now 1786091424642794 us; error 33 us; skew 500 ppm
I20260807 08:30:24.644425 16127 webserver.cc:460] Webserver started at http://127.1.157.126:39953/ using document root <none> and password file <none>
I20260807 08:30:24.644902 16127 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:24.644958 16127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:24.645107 16127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:24.646940 16127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "0c3e910d261646799de8e2057534b14b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "b1ab111c441e09def4da9fdcc6641f50"
server_key_iv: "3450d5b5686cf9f02d1d44873660e3c8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.647424 16127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "0c3e910d261646799de8e2057534b14b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "b1ab111c441e09def4da9fdcc6641f50"
server_key_iv: "3450d5b5686cf9f02d1d44873660e3c8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.650691 16127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:24.652798 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.653877 16127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:24.653982 16127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "0c3e910d261646799de8e2057534b14b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "b1ab111c441e09def4da9fdcc6641f50"
server_key_iv: "3450d5b5686cf9f02d1d44873660e3c8"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.654067 16127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:24.658210 16127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:24.658766 16127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:24.658916 16127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:24.665019 16127 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43125
I20260807 08:30:24.665023 16233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43125 every 8 connection(s)
I20260807 08:30:24.665872 16127 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:24.668731 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 16127
I20260807 08:30:24.668846 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:30:24.668862 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:24.669132 1653 external_mini_cluster.cc:1428] Setting key 9b813b366e3423f4def0b5f6ec4e357a
I20260807 08:30:24.674196 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.670383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.1.157.126:43125)
0807 08:30:24.670940 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.670954 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.670986 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:30:24.671755 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.671772 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.672081 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.672335 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.673096 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:24.673101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:24.673111 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:24.673121 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:24.673171 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:24.673186 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:24.673285 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:24.673387 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 08:30:24.673595 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260807 08:30:24.675011 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b: Bootstrap starting.
I20260807 08:30:24.677021 16234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:24.677735 16234 log.cc:824] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:24.679385 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b: No bootstrap required, opened a new log
I20260807 08:30:24.682516 16234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } }
I20260807 08:30:24.682708 16234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:24.682760 16234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c3e910d261646799de8e2057534b14b, State: Initialized, Role: FOLLOWER
I20260807 08:30:24.683111 16234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } }
I20260807 08:30:24.683233 16234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:24.683293 16234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:24.683380 16234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:24.684110 16234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } }
I20260807 08:30:24.684414 16234 leader_election.cc:302] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c3e910d261646799de8e2057534b14b; no voters:
I20260807 08:30:24.684688 16234 leader_election.cc:288] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:30:24.684779 16241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:24.684996 16241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [term 1 LEADER]: Becoming Leader. State: Replica: 0c3e910d261646799de8e2057534b14b, State: Running, Role: LEADER
I20260807 08:30:24.685336 16241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } }
I20260807 08:30:24.685704 16234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:24.686807 16244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c3e910d261646799de8e2057534b14b. Latest consensus state: current_term: 1 leader_uuid: "0c3e910d261646799de8e2057534b14b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } } }
I20260807 08:30:24.686781 16243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c3e910d261646799de8e2057534b14b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c3e910d261646799de8e2057534b14b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43125 } } }
I20260807 08:30:24.686990 16243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:24.687273 16244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:24.687364 16253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:24.689656 16253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:24.694142 16253 catalog_manager.cc:1347] Generated new cluster ID: 54700ab6807b4d568817ed618a8e2ed4
I20260807 08:30:24.694211 16253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:24.702929 16253 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:30:24.703171 16253 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:24.708798 16253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c3e910d261646799de8e2057534b14b: Generated new TSK 0
I20260807 08:30:24.709303 16253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:30:24.712744 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--builtin_ntp_servers=127.1.157.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 08:30:24.819387 16269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:24.819590 16269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:24.819628 16269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:24.823469 16269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:24.823511 16269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:24.823575 16269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:24.828101 16269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:24.828946 16269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:24.830133 16269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:30:24.838960 16269 server_base.cc:1034] running on GCE node
W20260807 08:30:24.838930 16279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.838912 16278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.838912 16282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.839534 16269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:24.840063 16269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:24.841212 16269 hybrid_clock.cc:648] HybridClock initialized: now 1786091424841197 us; error 25 us; skew 500 ppm
I20260807 08:30:24.842839 16269 webserver.cc:460] Webserver started at http://127.1.157.65:39193/ using document root <none> and password file <none>
I20260807 08:30:24.843313 16269 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:24.843369 16269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:24.843523 16269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:24.845368 16269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "836a644fcade4a6283d565d4d3412f1b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "ed666435015e18122d387bfdfee11a16"
server_key_iv: "1fea4c181a7a873a53854574c1e76bbe"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.845854 16269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "836a644fcade4a6283d565d4d3412f1b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "ed666435015e18122d387bfdfee11a16"
server_key_iv: "1fea4c181a7a873a53854574c1e76bbe"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.849190 16269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:30:24.851372 16289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.852424 16269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:24.852517 16269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "836a644fcade4a6283d565d4d3412f1b"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "ed666435015e18122d387bfdfee11a16"
server_key_iv: "1fea4c181a7a873a53854574c1e76bbe"
server_key_version: "encryptionkey@0"
I20260807 08:30:24.852597 16269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:24.856652 16269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:24.857152 16269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:24.857280 16269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:24.857777 16269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:24.858641 16269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:24.858698 16269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.858741 16269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:24.858760 16269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:24.866302 16269 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:45851
I20260807 08:30:24.866375 16462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:45851 every 8 connection(s)
I20260807 08:30:24.867089 16269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:24.868899 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 16269
I20260807 08:30:24.868993 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:30:24.869298 1653 external_mini_cluster.cc:1428] Setting key c74c4e1f2b743238071251d7d4cb303c
I20260807 08:30:24.871390 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--builtin_ntp_servers=127.1.157.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:30:24.872725 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.868598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:47979 (local address 127.1.157.126:43125)
0807 08:30:24.868736 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:24.868741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:24.869789 (+ 1048us) server_negotiation.cc:407] Connection header received
0807 08:30:24.871218 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:24.871220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:24.871253 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:24.871315 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:24.872364 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:24.872367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:24.872368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:24.872379 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:24.872399 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:24.872406 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:24.872473 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:24.872596 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:30:24.872628 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:30:24.873588 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:24.868930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43125 (local address 127.1.157.65:47979)
0807 08:30:24.869646 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:24.869699 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:24.871008 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:24.871482 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:24.871490 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:24.871597 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:24.872088 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:24.872090 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:24.872139 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:24.872220 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:24.872476 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:24.872484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:24.872489 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:30:24.872643 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:30:24.872872 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260807 08:30:24.874686 16463 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43125
I20260807 08:30:24.874882 16463 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:24.875222 16463 heartbeater.cc:511] Master 127.1.157.126:43125 requested a full tablet report, sending...
I20260807 08:30:24.876551 16173 ts_manager.cc:194] Registered new tserver with Master: 836a644fcade4a6283d565d4d3412f1b (127.1.157.65:45851)
W20260807 08:30:24.977368 16467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:24.977598 16467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:24.977650 16467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:24.981536 16467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:24.981582 16467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:24.981654 16467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:24.986203 16467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:24.987157 16467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:24.988402 16467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:24.997237 16475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.997242 16479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:24.997269 16473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:24.997752 16467 server_base.cc:1034] running on GCE node
I20260807 08:30:24.998104 16467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:24.998715 16467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:24.999871 16467 hybrid_clock.cc:648] HybridClock initialized: now 1786091424999855 us; error 27 us; skew 500 ppm
I20260807 08:30:25.001534 16467 webserver.cc:460] Webserver started at http://127.1.157.66:33463/ using document root <none> and password file <none>
I20260807 08:30:25.002043 16467 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:25.002102 16467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:25.002301 16467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:25.004284 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "ca10318c258a4e81805bd730eee05e51"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "33c18c2ac6e60317b2af2d055839ec22"
server_key_iv: "36bf0886bba0629b246ae8ef83dc9a68"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.004791 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "ca10318c258a4e81805bd730eee05e51"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "33c18c2ac6e60317b2af2d055839ec22"
server_key_iv: "36bf0886bba0629b246ae8ef83dc9a68"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.008231 16467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:30:25.010483 16486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.011575 16467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:25.011672 16467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "ca10318c258a4e81805bd730eee05e51"
format_stamp: "Formatted at 2026-08-07 08:30:24 on dist-test-slave-6zbq"
server_key: "33c18c2ac6e60317b2af2d055839ec22"
server_key_iv: "36bf0886bba0629b246ae8ef83dc9a68"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.011772 16467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:25.015991 16467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:25.016537 16467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:25.016682 16467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:25.017169 16467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:25.018011 16467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:25.018056 16467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.018095 16467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:25.018123 16467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.026032 16467 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:40119
I20260807 08:30:25.026090 16676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:40119 every 8 connection(s)
I20260807 08:30:25.026870 16467 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:25.027788 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 16467
I20260807 08:30:25.027894 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:30:25.028170 1653 external_mini_cluster.cc:1428] Setting key 19eba600eccc293d9885072f7213c608
I20260807 08:30:25.030128 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--builtin_ntp_servers=127.1.157.84:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 08:30:25.032644 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:25.028595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:55621 (local address 127.1.157.126:43125)
0807 08:30:25.028731 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:25.028735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:25.029799 (+ 1064us) server_negotiation.cc:407] Connection header received
0807 08:30:25.031127 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:25.031130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:25.031173 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:25.031245 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:25.032229 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:25.032233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:25.032235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:25.032248 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:25.032271 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:25.032279 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:25.032353 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:25.032475 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 08:30:25.032517 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:30:25.033376 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:25.028989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43125 (local address 127.1.157.66:55621)
0807 08:30:25.029665 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:25.029703 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:25.030927 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:25.031396 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:25.031403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:25.031481 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:25.031985 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:25.031986 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:25.032023 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:25.032088 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:25.032346 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:25.032354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:25.032360 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:30:25.032505 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 08:30:25.032718 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
I20260807 08:30:25.034373 16677 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43125
I20260807 08:30:25.034584 16677 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:25.034994 16677 heartbeater.cc:511] Master 127.1.157.126:43125 requested a full tablet report, sending...
I20260807 08:30:25.036026 16173 ts_manager.cc:194] Registered new tserver with Master: ca10318c258a4e81805bd730eee05e51 (127.1.157.66:40119)
W20260807 08:30:25.136994 16681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:25.137199 16681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:25.137236 16681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:25.141065 16681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:25.141124 16681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:25.141189 16681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:25.145630 16681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:25.146466 16681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:25.147800 16681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:25.156455 16693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:25.156461 16695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:25.156569 16681 server_base.cc:1034] running on GCE node
W20260807 08:30:25.156469 16697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:25.157120 16681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:25.157663 16681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:25.158820 16681 hybrid_clock.cc:648] HybridClock initialized: now 1786091425158801 us; error 37 us; skew 500 ppm
I20260807 08:30:25.160434 16681 webserver.cc:460] Webserver started at http://127.1.157.67:45817/ using document root <none> and password file <none>
I20260807 08:30:25.160948 16681 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:25.161012 16681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:25.161194 16681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:25.163096 16681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "0196c02e77424cadbbf1f8c4b0ccd072"
format_stamp: "Formatted at 2026-08-07 08:30:25 on dist-test-slave-6zbq"
server_key: "054a21c5ea9d309ef71adfc4c2e4baff"
server_key_iv: "2e67c4857d9702ab6e335e6f49b64618"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.163615 16681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "0196c02e77424cadbbf1f8c4b0ccd072"
format_stamp: "Formatted at 2026-08-07 08:30:25 on dist-test-slave-6zbq"
server_key: "054a21c5ea9d309ef71adfc4c2e4baff"
server_key_iv: "2e67c4857d9702ab6e335e6f49b64618"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.166889 16681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:30:25.169067 16705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.170133 16681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:30:25.170233 16681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "0196c02e77424cadbbf1f8c4b0ccd072"
format_stamp: "Formatted at 2026-08-07 08:30:25 on dist-test-slave-6zbq"
server_key: "054a21c5ea9d309ef71adfc4c2e4baff"
server_key_iv: "2e67c4857d9702ab6e335e6f49b64618"
server_key_version: "encryptionkey@0"
I20260807 08:30:25.170321 16681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:25.174428 16681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:25.174945 16681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:25.175100 16681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:25.175585 16681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:25.176376 16681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:25.176422 16681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.176467 16681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:25.176503 16681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:25.184229 16681 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39067
I20260807 08:30:25.184254 16890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39067 every 8 connection(s)
I20260807 08:30:25.184947 16681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:30:25.187156 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 16681
I20260807 08:30:25.187247 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:30:25.187507 1653 external_mini_cluster.cc:1428] Setting key 2f600befc0b71ab4dd30f5eee8ce90d5
I20260807 08:30:25.190711 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:25.186537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:33439 (local address 127.1.157.126:43125)
0807 08:30:25.186680 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:25.186684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:25.187986 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 08:30:25.189165 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:25.189169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:25.189212 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:25.189284 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:25.190274 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:25.190278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:25.190282 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:30:25.190299 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:25.190320 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:30:25.190327 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:25.190429 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:25.190549 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 08:30:25.190590 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:30:25.191431 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:25.187028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43125 (local address 127.1.157.67:33439)
0807 08:30:25.187849 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:25.187895 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:25.188977 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:25.189443 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:25.189450 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:25.189530 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:25.190030 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:30:25.190031 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:25.190067 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:30:25.190134 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:25.190418 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:25.190427 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:25.190433 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:30:25.190588 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 08:30:25.190807 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":147,"threads_started":1}
I20260807 08:30:25.192345 16891 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43125
I20260807 08:30:25.192533 16891 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:25.192867 16891 heartbeater.cc:511] Master 127.1.157.126:43125 requested a full tablet report, sending...
I20260807 08:30:25.193743 16173 ts_manager.cc:194] Registered new tserver with Master: 0196c02e77424cadbbf1f8c4b0ccd072 (127.1.157.67:39067)
I20260807 08:30:25.200681 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 08:30:25.203248 16238 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:30:25.202442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.1.157.126:43125)
0807 08:30:25.202542 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:25.202546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:25.202750 (+ 204us) server_negotiation.cc:407] Connection header received
0807 08:30:25.202851 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:25.202854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:25.202887 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:25.202946 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:25.203161 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36224: BlockingRecv error: recv got EOF from 127.0.0.1:36224 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:30:25.203547 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 16269
I20260807 08:30:25.224129 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 16467
I20260807 08:30:25.231770 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 16681
I20260807 08:30:25.239521 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 16127
2026-08-07T08:30:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (760 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:30:25 dist-test-slave-6zbq krb5kdc[16903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:30:25 dist-test-slave-6zbq krb5kdc[16903](info): set up 2 sockets
Aug 07 08:30:25 dist-test-slave-6zbq krb5kdc[16903](info): commencing operation
krb5kdc: starting...
W20260807 08:30:26.390247 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:30:26 dist-test-slave-6zbq krb5kdc[16903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:30:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:30:26.499032 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:32807
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:26.605746 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:26.606065 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:26.606120 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:26.610927 16928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:30:26.610980 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:26.610997 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:26.611012 16928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:30:26.611032 16928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:30:26.615972 16928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:32807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:32807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:26.616807 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:26.617669 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:26.625252 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:26.625285 16938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:26.625286 16940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:26.625392 16928 server_base.cc:1034] running on GCE node
I20260807 08:30:26.625962 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:26.626960 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:26.628111 16928 hybrid_clock.cc:648] HybridClock initialized: now 1786091426628094 us; error 31 us; skew 500 ppm
Aug 07 08:30:26 dist-test-slave-6zbq krb5kdc[16903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:26.635079 16928 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:30:26.635900 16928 webserver.cc:460] Webserver started at http://127.1.157.126:32951/ using document root <none> and password file <none>
I20260807 08:30:26.636392 16928 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:26.636451 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:26.636643 16928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:26.638612 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "047ff8e1f73b4b54811aba1245f30ef6"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "805a80716f0cf0556ee97e8a36f60002"
server_key_iv: "9332e3969c1093976dcf7c84c1b0f758"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.639142 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "047ff8e1f73b4b54811aba1245f30ef6"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "805a80716f0cf0556ee97e8a36f60002"
server_key_iv: "9332e3969c1093976dcf7c84c1b0f758"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.642648 16928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:30:26.644804 16955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:26.645895 16928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:30:26.645998 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "047ff8e1f73b4b54811aba1245f30ef6"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "805a80716f0cf0556ee97e8a36f60002"
server_key_iv: "9332e3969c1093976dcf7c84c1b0f758"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.646090 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:26.680887 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:26.683480 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:26.683642 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:26.690135 16928 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:32807
I20260807 08:30:26.690145 17039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:32807 every 8 connection(s)
I20260807 08:30:26.691051 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:30:26.693547 17042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:26.694289 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 16928
I20260807 08:30:26.694417 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:30:26.694736 1653 external_mini_cluster.cc:1428] Setting key aa70aa5b4526da7f44c354a01cdc2a28
I20260807 08:30:26.699229 17042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: Bootstrap starting.
I20260807 08:30:26.701330 17042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:26.702026 17042 log.cc:824] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:30:26 dist-test-slave-6zbq krb5kdc[16903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:26.703876 17042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: No bootstrap required, opened a new log
I20260807 08:30:26.707255 17042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } }
I20260807 08:30:26.707458 17042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:26.707515 17042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ff8e1f73b4b54811aba1245f30ef6, State: Initialized, Role: FOLLOWER
I20260807 08:30:26.707613 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:26.696090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54556 (local address 127.1.157.126:32807)
0807 08:30:26.696489 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:26.696503 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:26.696538 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:30:26.697147 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:26.697161 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:26.697468 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:26.697758 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:26.698572 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.699534 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.700377 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.700835 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.700979 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:26.703402 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:26.703407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:26.703420 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:26.703445 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:26.704993 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:26.705387 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:26.705392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:26.705396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:26.705504 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:26.705778 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:26.705781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:26.705783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:26.706139 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:26.706379 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:26.706633 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 08:30:26.706898 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":114,"threads_started":1}
I20260807 08:30:26.707957 17042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } }
I20260807 08:30:26.708112 17042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:30:26.708182 17042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:30:26.708271 17042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:26.709110 17042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } }
I20260807 08:30:26.709414 17042 leader_election.cc:302] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 047ff8e1f73b4b54811aba1245f30ef6; no voters:
I20260807 08:30:26.709695 17042 leader_election.cc:288] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:30:26.709781 17048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:26.710004 17048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [term 1 LEADER]: Becoming Leader. State: Replica: 047ff8e1f73b4b54811aba1245f30ef6, State: Running, Role: LEADER
I20260807 08:30:26.710368 17048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } }
I20260807 08:30:26.710920 17042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:30:26.711930 17050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "047ff8e1f73b4b54811aba1245f30ef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } } }
I20260807 08:30:26.712096 17050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:26.711993 17049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 047ff8e1f73b4b54811aba1245f30ef6. Latest consensus state: current_term: 1 leader_uuid: "047ff8e1f73b4b54811aba1245f30ef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047ff8e1f73b4b54811aba1245f30ef6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 32807 } } }
I20260807 08:30:26.712385 17049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:30:26.712482 17058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:30:26.715485 17058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:30:26.721233 17058 catalog_manager.cc:1347] Generated new cluster ID: 55fe4b1a6a4345c4acb4e6bcfab15462
I20260807 08:30:26.721303 17058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:30:26.736536 17058 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:30:26.737491 17058 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:30:26.746026 17058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: Generated new TSK 0
I20260807 08:30:26.746766 17058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:30:26.783330 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:26.888186 17079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:26.888412 17079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:26.888468 17079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:26.892331 17079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:26.892380 17079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:26.892457 17079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:30:26.897336 17079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:26.898213 17079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:26.899129 17079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:26.907835 17086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:26.907825 17088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:26.907824 17090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:26.908010 17079 server_base.cc:1034] running on GCE node
I20260807 08:30:26.908433 17079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:26.909015 17079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:26.910202 17079 hybrid_clock.cc:648] HybridClock initialized: now 1786091426910182 us; error 27 us; skew 500 ppm
Aug 07 08:30:26 dist-test-slave-6zbq krb5kdc[16903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:26.912777 17079 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:30:26.913579 17079 webserver.cc:460] Webserver started at http://127.1.157.65:45943/ using document root <none> and password file <none>
I20260807 08:30:26.914063 17079 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:26.914137 17079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:26.914318 17079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:26.916348 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "e445ec83bc324e4ab944d2106af14343"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "aaab68c3901aef98b47366ba6ed8e850"
server_key_iv: "cc48a2484ebce50efc93deaca9d3fca2"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.916868 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "e445ec83bc324e4ab944d2106af14343"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "aaab68c3901aef98b47366ba6ed8e850"
server_key_iv: "cc48a2484ebce50efc93deaca9d3fca2"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.920413 17079 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:30:26.922580 17098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:26.923668 17079 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:30:26.923772 17079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "e445ec83bc324e4ab944d2106af14343"
format_stamp: "Formatted at 2026-08-07 08:30:26 on dist-test-slave-6zbq"
server_key: "aaab68c3901aef98b47366ba6ed8e850"
server_key_iv: "cc48a2484ebce50efc93deaca9d3fca2"
server_key_version: "encryptionkey@0"
I20260807 08:30:26.923862 17079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:26.932989 17079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:26.935411 17079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:26.935564 17079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:26.936061 17079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:26.936789 17079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:26.936834 17079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:26.936884 17079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:26.936911 17079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:26.944845 17079 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:45409
I20260807 08:30:26.944867 17279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:45409 every 8 connection(s)
I20260807 08:30:26.945593 17079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:30:26.948787 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 17079
I20260807 08:30:26.948892 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:30:26.949132 1653 external_mini_cluster.cc:1428] Setting key 808142e9ba30c5b29e594c9044f2c27a
Aug 07 08:30:26 dist-test-slave-6zbq krb5kdc[16903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091426, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:30:26.958599 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:26.947195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40311 (local address 127.1.157.126:32807)
0807 08:30:26.947342 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:26.947347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:26.948070 (+ 723us) server_negotiation.cc:407] Connection header received
0807 08:30:26.949464 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:26.949468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:26.949523 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:26.949591 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:26.950947 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.951640 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.952758 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.952993 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.953083 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:26.955400 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:26.955403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:26.955405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:26.955428 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:26.956677 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:26.957138 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:26.957141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:26.957143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:26.957221 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:26.957530 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:26.957533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:26.957535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:26.957723 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:26.957809 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:26.958282 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 08:30:26.958436 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 08:30:26.959316 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:26.947523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:32807 (local address 127.1.157.65:40311)
0807 08:30:26.947935 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:26.947974 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:26.949261 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:26.949778 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:26.949787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:26.950237 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:26.950808 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:26.950822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.951756 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.951758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:26.952642 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:26.952652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:26.953152 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:26.953158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:26.953309 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:26.953829 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:26.953845 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:26.955248 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:26.956785 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:26.956791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:26.956806 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:26.957043 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:26.957304 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:26.957308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:26.957310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:26.957442 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:26.957824 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:26.957831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:26.958053 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 08:30:26.958220 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 08:30:26.958551 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":115,"threads_started":1}
I20260807 08:30:26.960415 17280 heartbeater.cc:347] Connected to a master server at 127.1.157.126:32807
I20260807 08:30:26.960613 17280 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:30:26.961094 17280 heartbeater.cc:511] Master 127.1.157.126:32807 requested a full tablet report, sending...
I20260807 08:30:26.962700 16979 ts_manager.cc:194] Registered new tserver with Master: e445ec83bc324e4ab944d2106af14343 (127.1.157.65:45409)
I20260807 08:30:26.963761 16979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:40311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:30:26.974181 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:27.079751 17287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:27.079973 17287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:27.080015 17287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:27.083953 17287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:27.084004 17287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:27.084070 17287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:30:27.089077 17287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:27.090020 17287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:27.090996 17287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:27.100173 17293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:27.100240 17287 server_base.cc:1034] running on GCE node
W20260807 08:30:27.100153 17297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:27.100391 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:27.100800 17287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:27.101409 17287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:27.102552 17287 hybrid_clock.cc:648] HybridClock initialized: now 1786091427102532 us; error 33 us; skew 500 ppm
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[16903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091427, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:27.105295 17287 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:30:27.106120 17287 webserver.cc:460] Webserver started at http://127.1.157.66:40679/ using document root <none> and password file <none>
I20260807 08:30:27.106693 17287 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:27.106760 17287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:27.106969 17287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:27.108968 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "9232b371cd47409fae07bfb5e6bcc683"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "a670e2c089644f2e394ef3e25a218287"
server_key_iv: "2a324a100178b0102a2df7b5d1e6d4df"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.109485 17287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "9232b371cd47409fae07bfb5e6bcc683"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "a670e2c089644f2e394ef3e25a218287"
server_key_iv: "2a324a100178b0102a2df7b5d1e6d4df"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.113107 17287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 08:30:27.115355 17307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.116425 17287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:27.116539 17287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "9232b371cd47409fae07bfb5e6bcc683"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "a670e2c089644f2e394ef3e25a218287"
server_key_iv: "2a324a100178b0102a2df7b5d1e6d4df"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.116626 17287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:27.132493 17287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:27.135847 17287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:27.136022 17287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:27.136577 17287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:27.137365 17287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:27.137413 17287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.137465 17287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:27.137492 17287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.145588 17287 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:44489
I20260807 08:30:27.145644 17497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:44489 every 8 connection(s)
I20260807 08:30:27.146466 17287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:30:27.149468 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 17287
I20260807 08:30:27.149590 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:30:27.149966 1653 external_mini_cluster.cc:1428] Setting key 8c5ac8eaa34e65041364d9c8700ba8ad
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[16903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091427, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:27.160365 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.148128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:44693 (local address 127.1.157.126:32807)
0807 08:30:27.148270 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.148274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.149053 (+ 779us) server_negotiation.cc:407] Connection header received
0807 08:30:27.150697 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.150708 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.150764 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.150837 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:27.152163 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.152675 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.153965 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.154195 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.154285 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.156892 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:27.156896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:27.156899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:27.156930 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:27.158460 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.158918 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.158922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.158924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.159018 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.159306 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.159309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.159311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.159526 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:27.159633 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.160068 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.160203 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:30:27.161068 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.148478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:32807 (local address 127.1.157.66:44693)
0807 08:30:27.148919 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.148960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.150503 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.151011 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.151017 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.151382 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:27.152013 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.152031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.152803 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.152807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.153848 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.153858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.154334 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.154359 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.154514 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.155012 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:27.155028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:27.156722 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:27.158562 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:27.158567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:27.158577 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:27.158825 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:27.159100 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:27.159102 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:27.159103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:27.159223 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:27.159616 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:27.159620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:27.159843 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.159987 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.160222 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260807 08:30:27.162153 17498 heartbeater.cc:347] Connected to a master server at 127.1.157.126:32807
I20260807 08:30:27.162410 17498 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:30:27.162902 17498 heartbeater.cc:511] Master 127.1.157.126:32807 requested a full tablet report, sending...
I20260807 08:30:27.163962 16979 ts_manager.cc:194] Registered new tserver with Master: 9232b371cd47409fae07bfb5e6bcc683 (127.1.157.66:44489)
I20260807 08:30:27.164667 16979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:44693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:30:27.175442 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:30:27.286705 17503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:30:27.286972 17503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:30:27.287034 17503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:30:27.291038 17503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:30:27.291101 17503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:30:27.291189 17503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:30:27.296420 17503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:30:27.297551 17503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:30:27.298635 17503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:30:27.308363 17511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:27.308579 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:30:27.308622 17516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:30:27.308655 17503 server_base.cc:1034] running on GCE node
I20260807 08:30:27.309067 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:30:27.309686 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:30:27.310853 17503 hybrid_clock.cc:648] HybridClock initialized: now 1786091427310835 us; error 34 us; skew 500 ppm
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[16903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091427, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:30:27.313607 17503 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:30:27.314523 17503 webserver.cc:460] Webserver started at http://127.1.157.67:43717/ using document root <none> and password file <none>
I20260807 08:30:27.315050 17503 fs_manager.cc:359] Metadata directory not provided
I20260807 08:30:27.315111 17503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:30:27.315318 17503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:30:27.317333 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "6da36460f03a4fb4ab23dbf7e914df93"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "54c862cfa21fcc2d9ff1c49fefa51326"
server_key_iv: "3c91972c44b939fae69e9ebd9428c5c7"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.317870 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "6da36460f03a4fb4ab23dbf7e914df93"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "54c862cfa21fcc2d9ff1c49fefa51326"
server_key_iv: "3c91972c44b939fae69e9ebd9428c5c7"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.321568 17503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:30:27.323887 17529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.325140 17503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:30:27.325264 17503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "6da36460f03a4fb4ab23dbf7e914df93"
format_stamp: "Formatted at 2026-08-07 08:30:27 on dist-test-slave-6zbq"
server_key: "54c862cfa21fcc2d9ff1c49fefa51326"
server_key_iv: "3c91972c44b939fae69e9ebd9428c5c7"
server_key_version: "encryptionkey@0"
I20260807 08:30:27.325362 17503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:30:27.351570 17503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:30:27.354462 17503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:30:27.354633 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:30:27.355213 17503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:30:27.356014 17503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:30:27.356062 17503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.356124 17503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:30:27.356151 17503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:30:27.364392 17503 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:46609
I20260807 08:30:27.364424 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:46609 every 8 connection(s)
I20260807 08:30:27.365221 17503 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:30:27.371444 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 17503
I20260807 08:30:27.371552 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:30:27.371796 1653 external_mini_cluster.cc:1428] Setting key 7ee248e58835e607b5dbeeb5c58f390c
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[16903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091427, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:30:27.377666 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.366917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:43595 (local address 127.1.157.126:32807)
0807 08:30:27.367051 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.367055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.367865 (+ 810us) server_negotiation.cc:407] Connection header received
0807 08:30:27.369140 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.369142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.369200 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.369262 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:27.370323 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.370836 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.371999 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.372259 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.372360 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.374923 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:27.374926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:27.374928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:27.374954 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:27.376049 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.376479 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.376481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.376482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.376549 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.376794 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.376796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.376797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.376953 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:27.377019 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.377446 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.377543 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:30:27.378450 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.367297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:32807 (local address 127.1.157.67:43595)
0807 08:30:27.367716 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.367763 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.368945 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.369404 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.369410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.369763 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:30:27.370214 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.370227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.370932 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.370935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.371877 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.371888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.372403 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.372407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.372622 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.373236 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:30:27.373255 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:30:27.374772 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:30:27.376148 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:27.376153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:27.376164 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:27.376401 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:27.376609 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:30:27.376611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:30:27.376613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:30:27.376733 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:30:27.377019 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:30:27.377023 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:30:27.377252 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.377396 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.377630 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":112,"threads_started":1}
I20260807 08:30:27.379473 17718 heartbeater.cc:347] Connected to a master server at 127.1.157.126:32807
I20260807 08:30:27.379688 17718 heartbeater.cc:464] Registering TS with master...
I20260807 08:30:27.380153 17718 heartbeater.cc:511] Master 127.1.157.126:32807 requested a full tablet report, sending...
I20260807 08:30:27.381099 16979 ts_manager.cc:194] Registered new tserver with Master: 6da36460f03a4fb4ab23dbf7e914df93 (127.1.157.67:46609)
I20260807 08:30:27.381647 16979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:43595
I20260807 08:30:27.385283 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:30:27.393509 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.387072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.1.157.126:32807)
0807 08:30:27.387280 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.387283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.387358 (+ 75us) server_negotiation.cc:407] Connection header received
0807 08:30:27.387456 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.387458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.387491 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.387558 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:30:27.388267 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.388758 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.389553 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.389763 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.389843 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.391029 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:30:27.391032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:30:27.391034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:30:27.391055 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:30:27.392280 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.392622 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.392625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.392626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.392692 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:30:27.392874 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:30:27.392876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:30:27.392877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:30:27.393064 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:30:27.393174 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.393314 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.393399 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 08:30:27.396070 16979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:30:27.399022 16979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:30:27.412102 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.406942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:46609 (local address 127.0.0.1:52772)
0807 08:30:27.407204 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.407217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.407305 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.408332 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.408335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.408352 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:27.408448 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.408454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409464 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.409467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.410664 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.410675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.411606 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.411609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.411724 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.411764 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.411857 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.411968 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
I20260807 08:30:27.412117 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.407183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.1.157.67:46609)
0807 08:30:27.407939 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.407952 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.407972 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:30:27.408041 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.408046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.408207 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.408340 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:27.408569 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409346 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.410802 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.411494 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.411590 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.411725 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.411899 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.411992 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":118,"threads_started":1}
I20260807 08:30:27.412552 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.406528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44489 (local address 127.0.0.1:52394)
0807 08:30:27.406820 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.406873 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.406986 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.408059 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.408065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.408089 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:27.408212 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.408219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409860 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.409864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.411072 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.411081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.412154 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.412158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.412274 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.412354 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.412402 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.412454 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":82,"thread_start_us":61,"threads_started":1}
I20260807 08:30:27.413053 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.406708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.1.157.66:44489)
0807 08:30:27.407551 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.407564 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.407587 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:30:27.407664 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.407669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.407875 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.408378 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:27.408547 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409717 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.411204 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.412005 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.412741 (+ 736us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.412850 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.412889 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.412948 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":115,"threads_started":1}
I20260807 08:30:27.413475 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.407186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:45409 (local address 127.0.0.1:59572)
0807 08:30:27.407629 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.407641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.407720 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.408493 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.408496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.408512 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:27.408583 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.408588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409905 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.409908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.410959 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.410969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.412220 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.412224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.412311 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.413194 (+ 883us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.413283 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.413353 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":873,"thread_start_us":91,"threads_started":1}
I20260807 08:30:27.413857 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.407599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.1.157.65:45409)
0807 08:30:27.408021 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.408034 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.408054 (+ 20us) server_negotiation.cc:407] Connection header received
0807 08:30:27.408124 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.408129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.408315 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.408447 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:27.408695 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.409781 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.411074 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.411909 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.412014 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.412151 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.413631 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.413723 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":111,"threads_started":1}
I20260807 08:30:27.415292 17615 tablet_service.cc:1467] Processing CreateTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 (DEFAULT_TABLE table=test-table [id=d9d9358bc133431d87817c64d4aba1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:27.415534 17392 tablet_service.cc:1467] Processing CreateTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 (DEFAULT_TABLE table=test-table [id=d9d9358bc133431d87817c64d4aba1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:27.416371 17392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9e28ea78724379afa5bd629bce9fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:27.416388 17615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9e28ea78724379afa5bd629bce9fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:27.416587 17180 tablet_service.cc:1467] Processing CreateTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 (DEFAULT_TABLE table=test-table [id=d9d9358bc133431d87817c64d4aba1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:30:27.417506 17180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce9e28ea78724379afa5bd629bce9fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:30:27.422380 17744 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Bootstrap starting.
I20260807 08:30:27.424199 17745 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Bootstrap starting.
I20260807 08:30:27.424377 17746 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Bootstrap starting.
I20260807 08:30:27.425370 17744 tablet_bootstrap.cc:654] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:27.426218 17744 log.cc:824] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:27.426362 17745 tablet_bootstrap.cc:654] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:27.426453 17746 tablet_bootstrap.cc:654] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Neither blocks nor log segments found. Creating new log.
I20260807 08:30:27.427170 17745 log.cc:824] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:27.427556 17746 log.cc:824] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Log is configured to *not* fsync() on all Append() calls
I20260807 08:30:27.428428 17744 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: No bootstrap required, opened a new log
I20260807 08:30:27.428622 17744 ts_tablet_manager.cc:1397] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:30:27.428987 17745 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: No bootstrap required, opened a new log
I20260807 08:30:27.429230 17745 ts_tablet_manager.cc:1397] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:30:27.429791 17746 tablet_bootstrap.cc:492] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: No bootstrap required, opened a new log
I20260807 08:30:27.429963 17746 ts_tablet_manager.cc:1397] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 08:30:27.432245 17745 raft_consensus.cc:348] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.432442 17745 raft_consensus.cc:374] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:27.432520 17745 raft_consensus.cc:729] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da36460f03a4fb4ab23dbf7e914df93, State: Initialized, Role: FOLLOWER
I20260807 08:30:27.432397 17744 raft_consensus.cc:348] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.432638 17744 raft_consensus.cc:374] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:27.432693 17744 raft_consensus.cc:729] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9232b371cd47409fae07bfb5e6bcc683, State: Initialized, Role: FOLLOWER
I20260807 08:30:27.432929 17745 consensus_queue.cc:260] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.433092 17744 consensus_queue.cc:260] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.433575 17718 heartbeater.cc:503] Master 127.1.157.126:32807 was elected leader, sending a full tablet report...
I20260807 08:30:27.433859 17745 ts_tablet_manager.cc:1428] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:30:27.433703 17746 raft_consensus.cc:348] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.433944 17746 raft_consensus.cc:374] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:30:27.433995 17746 raft_consensus.cc:729] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e445ec83bc324e4ab944d2106af14343, State: Initialized, Role: FOLLOWER
I20260807 08:30:27.434015 17744 ts_tablet_manager.cc:1428] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:30:27.434388 17746 consensus_queue.cc:260] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.434499 17498 heartbeater.cc:503] Master 127.1.157.126:32807 was elected leader, sending a full tablet report...
I20260807 08:30:27.434984 17280 heartbeater.cc:503] Master 127.1.157.126:32807 was elected leader, sending a full tablet report...
I20260807 08:30:27.435144 17746 ts_tablet_manager.cc:1428] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 08:30:27.446588 17281 tablet.cc:2369] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:30:27.596906 17755 raft_consensus.cc:482] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:30:27.597126 17755 raft_consensus.cc:504] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.598290 17755 leader_election.cc:288] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9232b371cd47409fae07bfb5e6bcc683 (127.1.157.66:44489), 6da36460f03a4fb4ab23dbf7e914df93 (127.1.157.67:46609)
I20260807 08:30:27.602113 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.598703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:52095 (local address 127.1.157.67:46609)
0807 08:30:27.598915 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.598919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.598967 (+ 48us) server_negotiation.cc:407] Connection header received
0807 08:30:27.599113 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.599116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.599169 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.599236 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:27.599516 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.600007 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601183 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.601563 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601660 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.601713 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.601928 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.601995 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 08:30:27.602118 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.598672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:34881 (local address 127.1.157.66:44489)
0807 08:30:27.598806 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.598810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.598970 (+ 160us) server_negotiation.cc:407] Connection header received
0807 08:30:27.599101 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.599104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.599155 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.599219 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:30:27.599530 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.600015 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601193 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.601579 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601685 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.601744 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.601946 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.602012 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:30:27.602134 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.598526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44489 (local address 127.1.157.65:34881)
0807 08:30:27.598866 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.598880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.598992 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.599248 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.599251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.599267 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:27.599422 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.599427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.600123 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.600126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.601056 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.601065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.601699 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.601831 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.601848 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.601916 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.601983 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":111,"spinlock_wait_cycles":23424,"thread_start_us":87,"threads_started":1}
I20260807 08:30:27.602138 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.598559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:46609 (local address 127.1.157.65:52095)
0807 08:30:27.598891 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.598904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:30:27.599011 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:30:27.599259 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:30:27.599262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:30:27.599278 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:30:27.599412 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.599417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.600114 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.600118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.601045 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:30:27.601055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.601678 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.601682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:30:27.601776 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.601833 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 08:30:27.601894 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:30:27.601966 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":68,"thread_start_us":150,"threads_started":1}
I20260807 08:30:27.602944 17650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9e28ea78724379afa5bd629bce9fa9" candidate_uuid: "e445ec83bc324e4ab944d2106af14343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da36460f03a4fb4ab23dbf7e914df93" is_pre_election: true
I20260807 08:30:27.602968 17428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9e28ea78724379afa5bd629bce9fa9" candidate_uuid: "e445ec83bc324e4ab944d2106af14343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9232b371cd47409fae07bfb5e6bcc683" is_pre_election: true
I20260807 08:30:27.603220 17428 raft_consensus.cc:2454] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e445ec83bc324e4ab944d2106af14343 in term 0.
I20260807 08:30:27.603246 17650 raft_consensus.cc:2454] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e445ec83bc324e4ab944d2106af14343 in term 0.
I20260807 08:30:27.603672 17099 leader_election.cc:302] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6da36460f03a4fb4ab23dbf7e914df93, e445ec83bc324e4ab944d2106af14343; no voters:
I20260807 08:30:27.603930 17755 raft_consensus.cc:2781] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:30:27.604017 17755 raft_consensus.cc:482] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:30:27.604064 17755 raft_consensus.cc:3037] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:27.605039 17755 raft_consensus.cc:504] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.605332 17755 leader_election.cc:288] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [CANDIDATE]: Term 1 election: Requested vote from peers 9232b371cd47409fae07bfb5e6bcc683 (127.1.157.66:44489), 6da36460f03a4fb4ab23dbf7e914df93 (127.1.157.67:46609)
I20260807 08:30:27.605624 17428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9e28ea78724379afa5bd629bce9fa9" candidate_uuid: "e445ec83bc324e4ab944d2106af14343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9232b371cd47409fae07bfb5e6bcc683"
I20260807 08:30:27.605665 17650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce9e28ea78724379afa5bd629bce9fa9" candidate_uuid: "e445ec83bc324e4ab944d2106af14343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da36460f03a4fb4ab23dbf7e914df93"
I20260807 08:30:27.605732 17428 raft_consensus.cc:3037] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:27.605773 17650 raft_consensus.cc:3037] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:30:27.606757 17428 raft_consensus.cc:2454] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e445ec83bc324e4ab944d2106af14343 in term 1.
I20260807 08:30:27.606797 17650 raft_consensus.cc:2454] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e445ec83bc324e4ab944d2106af14343 in term 1.
I20260807 08:30:27.607012 17104 leader_election.cc:302] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9232b371cd47409fae07bfb5e6bcc683, e445ec83bc324e4ab944d2106af14343; no voters:
I20260807 08:30:27.607226 17755 raft_consensus.cc:2781] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:30:27.607429 17755 raft_consensus.cc:686] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [term 1 LEADER]: Becoming Leader. State: Replica: e445ec83bc324e4ab944d2106af14343, State: Running, Role: LEADER
I20260807 08:30:27.607731 17755 consensus_queue.cc:237] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } }
I20260807 08:30:27.610623 16976 catalog_manager.cc:5495] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 reported cstate change: term changed from 0 to 1, leader changed from <none> to e445ec83bc324e4ab944d2106af14343 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "e445ec83bc324e4ab944d2106af14343" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e445ec83bc324e4ab944d2106af14343" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 45409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 } health_report { overall_health: UNKNOWN } } }
W20260807 08:30:27.616102 17719 tablet.cc:2369] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:30:27.645177 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.641929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.1.157.65:45409)
0807 08:30:27.642039 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:30:27.642043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:30:27.642056 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:30:27.642188 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:30:27.642203 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:30:27.642254 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:30:27.642370 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:30:27.642591 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.643053 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.643904 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:30:27.644106 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:30:27.644188 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:30:27.644472 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:30:27.644837 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:30:27.644938 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:30:27.645009 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:30:27.645057 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 08:30:27.647445 17499 tablet.cc:2369] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:30:27.650478 17650 raft_consensus.cc:1261] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 1 FOLLOWER]: Refusing update from remote peer e445ec83bc324e4ab944d2106af14343: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:27.650478 17428 raft_consensus.cc:1261] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 1 FOLLOWER]: Refusing update from remote peer e445ec83bc324e4ab944d2106af14343: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:30:27.651113 17755 consensus_queue.cc:1035] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9232b371cd47409fae07bfb5e6bcc683" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:27.651302 17761 consensus_queue.cc:1035] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6da36460f03a4fb4ab23dbf7e914df93" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 46609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:30:27.658463 17766 mvcc.cc:205] Tried to move back new op lower bound from 7315830487649673216 to 7315830487483604992. Current Snapshot: MvccSnapshot[applied={T|T < 7315830487649673216}]
I20260807 08:30:27.660236 17767 mvcc.cc:205] Tried to move back new op lower bound from 7315830487649673216 to 7315830487483604992. Current Snapshot: MvccSnapshot[applied={T|T < 7315830487649673216}]
I20260807 08:30:27.666572 16976 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:27.666709 16976 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:30:27.668742 16976 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 047ff8e1f73b4b54811aba1245f30ef6: Sending DeleteTablet for 3 replicas of tablet ce9e28ea78724379afa5bd629bce9fa9
I20260807 08:30:27.669391 17615 tablet_service.cc:1514] Processing DeleteTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:52772
I20260807 08:30:27.669466 17392 tablet_service.cc:1514] Processing DeleteTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:52394
I20260807 08:30:27.669605 17180 tablet_service.cc:1514] Processing DeleteTablet for tablet ce9e28ea78724379afa5bd629bce9fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:30:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:59572
I20260807 08:30:27.670009 17778 tablet_replica.cc:330] T ce9e28ea78724379afa5bd629bce9fa9 P e445ec83bc324e4ab944d2106af14343: stopping tablet replica
I20260807 08:30:27.670022 17777 tablet_replica.cc:330] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: stopping tablet replica
I20260807 08:30:27.670287 17777 raft_consensus.cc:2229] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:27.670420 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 17079
I20260807 08:30:27.670578 17777 raft_consensus.cc:2258] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:27.674131 17776 tablet_replica.cc:330] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: stopping tablet replica
I20260807 08:30:27.674484 17776 raft_consensus.cc:2229] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:30:27.674834 17776 raft_consensus.cc:2258] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:30:27.679005 17776 ts_tablet_manager.cc:1905] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:27.682183 17776 ts_tablet_manager.cc:1918] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:27.682318 17776 log.cc:1196] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/ce9e28ea78724379afa5bd629bce9fa9
I20260807 08:30:27.682675 17776 ts_tablet_manager.cc:1939] T ce9e28ea78724379afa5bd629bce9fa9 P 9232b371cd47409fae07bfb5e6bcc683: Deleting consensus metadata
I20260807 08:30:27.683780 16958 catalog_manager.cc:4841] TS 9232b371cd47409fae07bfb5e6bcc683 (127.1.157.66:44489): tablet ce9e28ea78724379afa5bd629bce9fa9 (table test-table [id=d9d9358bc133431d87817c64d4aba1e3]) successfully deleted
I20260807 08:30:27.690939 17777 ts_tablet_manager.cc:1905] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:30:27.693382 17777 ts_tablet_manager.cc:1918] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:30:27.693470 17777 log.cc:1196] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/ce9e28ea78724379afa5bd629bce9fa9
I20260807 08:30:27.693794 17777 ts_tablet_manager.cc:1939] T ce9e28ea78724379afa5bd629bce9fa9 P 6da36460f03a4fb4ab23dbf7e914df93: Deleting consensus metadata
I20260807 08:30:27.694872 16957 catalog_manager.cc:4841] TS 6da36460f03a4fb4ab23dbf7e914df93 (127.1.157.67:46609): tablet ce9e28ea78724379afa5bd629bce9fa9 (table test-table [id=d9d9358bc133431d87817c64d4aba1e3]) successfully deleted
W20260807 08:30:27.695186 16958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:45409 (error 108)
I20260807 08:30:27.695255 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 17287
I20260807 08:30:27.699263 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:30:27.699006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:45409 (local address 127.0.0.1:59594)
0807 08:30:27.699121 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 08:30:27.699187 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 08:30:27.699424 16958 catalog_manager.cc:4584] TS e445ec83bc324e4ab944d2106af14343 (127.1.157.65:45409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce9e28ea78724379afa5bd629bce9fa9: Network error: Client connection negotiation failed: client connection to 127.1.157.65:45409: connect: Connection refused (error 111)
I20260807 08:30:27.704335 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 17503
I20260807 08:30:27.723865 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 16928
2026-08-07T08:30:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2483 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[17789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[17789](info): set up 2 sockets
Aug 07 08:30:27 dist-test-slave-6zbq krb5kdc[17789](info): commencing operation
krb5kdc: starting...
W20260807 08:30:28.847240 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:30:28 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:30:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:30:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangeradmin_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangeradmin_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangerlookup_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangerlookup_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
I20260807 08:30:29.002187 1653 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksV4IkV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/postgres -l logfile start
2026-08-07 08:30:31.202 UTC [17836] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:30:31.202 UTC [17836] LOG: listening on IPv4 address "127.1.157.84", port 41961
2026-08-07 08:30:31.205 UTC [17836] LOG: listening on Unix socket "/tmp/.s.PGSQL.41961"
2026-08-07 08:30:31.224 UTC [17840] LOG: database system was shut down at 2026-08-07 08:30:30 UTC
2026-08-07 08:30:31.227 UTC [17836] LOG: database system is ready to accept connections
I20260807 08:30:32.278051 1653 mini_postgres.cc:96] Postgres bound to 41961
2026-08-07 08:30:32.285 UTC [17853] FATAL: database "slave" does not exist
127.1.157.84:41961 - accepting connections
I20260807 08:30:32.286563 1653 mini_ranger.cc:161] Starting Ranger...
I20260807 08:30:32.301519 1653 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:30:32.690295 1653 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:30:32.690415 1653 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin
2026-08-07 08:30:32,993 [I] DB FLAVOR :POSTGRES
2026-08-07 08:30:32,993 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:30:32,993 [I] Checking connection..
2026-08-07 08:30:32,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:30:33,253 [I] Checking connection passed.
2026-08-07 08:30:33,253 [I] --------- Verifying version history table ---------
2026-08-07 08:30:33,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:30:33,489 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:30:33,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:30:33,717 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:30:33,718 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:30:33,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:30:33.968 UTC [17962] WARNING: there is no transaction in progress
2026-08-07 08:30:33,975 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:30:33,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:30:34,225 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:30:34,225 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:30:34,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:30:34,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:30:34,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-07 08:30:34,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:30:35,208 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:30:35,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:30:35,460 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:30:35,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:30:35,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:30:35,936 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:30:35,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:30:36.787 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.801 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.814 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.827 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.842 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.856 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.906 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.913 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.925 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.934 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.943 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.955 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.964 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.970 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.975 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:36.980 UTC [18142] WARNING: there is no transaction in progress
2026-08-07 08:30:37,508 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:30:37,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:30:37,756 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:30:37,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:30:38,009 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:30:38,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:30:38,239 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:30:38,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:30:38,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:30:38,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-07 08:30:38,957 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:30:38,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:30:39,187 [I] DB_PATCHES have already been applied
I20260807 08:30:39.193287 1653 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksV4IkV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:30:39.193344 1653 mini_ranger.cc:193] Using host: 127.1.157.84
I20260807 08:30:39.197486 1653 mini_ranger.cc:227] Ranger admin URL: http://127.1.157.84:47901
Aug 07, 2026 8:30:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:30:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:30:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:30:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:30:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:30:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:30:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:30:40.403255 1653 mini_ranger.cc:160] Time spent starting Ranger: real 8.117s user 0.000s sys 0.011s
Aug 07, 2026 8:30:59 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:30:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 8:30:59 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:30:59 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:31:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:31:01.163233 1653 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangerkms_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/rangerkms_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.157.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.157.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:31:01 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091461, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:31:01.247825 1653 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:31:01.260603 1653 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:31:01.932646 1653 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:31:01.932744 1653 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms
I20260807 08:31:01.932835 1653 mini_ranger_kms.cc:227] Using postgres at 127.1.157.84:41961
2026-08-07 08:31:02,669 [I] DB FLAVOR :POSTGRES
2026-08-07 08:31:02,670 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:31:02,670 [I] Checking connection
2026-08-07 08:31:02,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:31:02,971 [I] connection success
2026-08-07 08:31:02,972 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:31:02,972 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:31:02,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:31:03,237 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:31:03,238 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:31:03,238 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:31:03,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:41961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:31:03,539 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:31:03.943931 1653 mini_ranger_kms.cc:310] Created kms service
I20260807 08:31:04.115288 1653 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:31:04.154552 1653 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:31:04 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091461, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:04.509120 1653 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:31:04.509260 1653 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:31:04.509290 1653 mini_ranger_kms.cc:240] Using host: 127.1.157.84
I20260807 08:31:04.514611 1653 mini_ranger_kms.cc:276] Ranger KMS PID: 18418
I20260807 08:31:04.514750 1653 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.157.84:47383
Aug 07, 2026 8:31:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:31:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:31:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:31:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:31:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:31:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 08:31:07.069187 1653 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.821s user 0.029s sys 0.562s
I20260807 08:31:07.069332 1653 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:31:07.069450 1653 mini_ranger_kms.cc:401] 127.1.157.84:47383/kms/v1/keys
Aug 07 08:31:07 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 08:31:07 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
Aug 07 08:31:07 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
Aug 07 08:31:10 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:31:10.246395 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:36069
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:36069
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:47383/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:10.379587 18492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:10.379822 18492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:10.379859 18492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:10.383774 18492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:31:10.383826 18492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:10.383843 18492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:10.383857 18492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:31:10.383876 18492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:31:10.389259 18492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.157.84:47383/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:36069
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:36069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:10.390177 18492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:10.391155 18492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:10.399243 18497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:10.399288 18499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:10.399361 18492 server_base.cc:1034] running on GCE node
W20260807 08:31:10.399289 18504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:10.400045 18492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:10.401063 18492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:10.402222 18492 hybrid_clock.cc:648] HybridClock initialized: now 1786091470402205 us; error 30 us; skew 500 ppm
Aug 07 08:31:10 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:10.408008 18492 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:31:10.408809 18492 webserver.cc:460] Webserver started at http://127.1.157.126:43769/ using document root <none> and password file <none>
I20260807 08:31:10.409363 18492 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:10.409420 18492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:10.409587 18492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:10 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:10.467379 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "885d9e7fd5214081a89ed29c7384473f"
format_stamp: "Formatted at 2026-08-07 08:31:10 on dist-test-slave-6zbq"
server_key: "f9e739061f2d477dab094746e496a469"
server_key_iv: "926b9fe670a1d294892bc4462731fd8d"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:10.468128 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "885d9e7fd5214081a89ed29c7384473f"
format_stamp: "Formatted at 2026-08-07 08:31:10 on dist-test-slave-6zbq"
server_key: "f9e739061f2d477dab094746e496a469"
server_key_iv: "926b9fe670a1d294892bc4462731fd8d"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:10.473634 18492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 08:31:10.490147 18521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:10.491743 18492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:31:10.491952 18492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "885d9e7fd5214081a89ed29c7384473f"
format_stamp: "Formatted at 2026-08-07 08:31:10 on dist-test-slave-6zbq"
server_key: "f9e739061f2d477dab094746e496a469"
server_key_iv: "926b9fe670a1d294892bc4462731fd8d"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:10.492202 18492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:10.520150 18492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:10.523224 18492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:10.523406 18492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:10.537839 18492 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:36069
I20260807 08:31:10.537899 18600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:36069 every 8 connection(s)
I20260807 08:31:10.539182 18492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:31:10.542016 18602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:10.543111 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 18492
I20260807 08:31:10.543241 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:31:10.543550 1653 external_mini_cluster.cc:1428] Setting key d3cd132c35076d5781236d6ccebc8e43
I20260807 08:31:10.549165 18602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: Bootstrap starting.
I20260807 08:31:10.551987 18602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:10.553166 18602 log.cc:824] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:31:10 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:10.555423 18602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: No bootstrap required, opened a new log
I20260807 08:31:10.560042 18602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } }
I20260807 08:31:10.560304 18602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:10.560390 18602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885d9e7fd5214081a89ed29c7384473f, State: Initialized, Role: FOLLOWER
I20260807 08:31:10.560839 18602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } }
I20260807 08:31:10.560974 18602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:31:10.561054 18602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:31:10.561165 18602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:10.561409 18610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:10.544979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.1.157.126:36069)
0807 08:31:10.545510 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:10.545529 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:10.545574 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:31:10.546660 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:10.546685 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:10.547128 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:10.548394 (+ 1266us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:10.548804 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:10.550254 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:10.551505 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:10.552286 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:10.552450 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:10.555604 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:10.555615 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:10.555633 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:10.555667 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:10.557766 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:10.558260 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:10.558266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:10.558272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:10.558451 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:10.558803 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:10.558807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:10.558808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:10.559459 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:10.559657 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:10.560014 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 08:31:10.560453 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":149,"threads_started":1}
I20260807 08:31:10.562230 18602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } }
I20260807 08:31:10.562579 18602 leader_election.cc:302] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 885d9e7fd5214081a89ed29c7384473f; no voters:
I20260807 08:31:10.562929 18602 leader_election.cc:288] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:31:10.563024 18615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:31:10.563308 18615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [term 1 LEADER]: Becoming Leader. State: Replica: 885d9e7fd5214081a89ed29c7384473f, State: Running, Role: LEADER
I20260807 08:31:10.564146 18602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:31:10.564167 18615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } }
I20260807 08:31:10.565688 18615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 885d9e7fd5214081a89ed29c7384473f. Latest consensus state: current_term: 1 leader_uuid: "885d9e7fd5214081a89ed29c7384473f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } } }
I20260807 08:31:10.565824 18615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:10.566216 18633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:31:10.567034 18619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "885d9e7fd5214081a89ed29c7384473f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d9e7fd5214081a89ed29c7384473f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 36069 } } }
I20260807 08:31:10.567143 18619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:10.570277 18633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:31:10.570791 18602 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:31:10.578590 18633 catalog_manager.cc:1347] Generated new cluster ID: 0e8731445b124416bbfd4f59fe834ddc
I20260807 08:31:10.578653 18633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:31:10.619503 18633 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:31:10.620867 18633 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:31:10.625888 18633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: Generated new TSK 0
I20260807 08:31:10.626857 18633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:31:11.430832 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:47383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:11.571007 18675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:11.571255 18675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:11.571317 18675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:11.575760 18675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:11.575816 18675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:11.575891 18675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:31:11.581068 18675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.157.84:47383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:11.582036 18675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:11.583065 18675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:11.594507 18688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:11.594568 18689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:11.594511 18692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:11.595136 18675 server_base.cc:1034] running on GCE node
I20260807 08:31:11.595595 18675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:11.596349 18675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:11.597539 18675 hybrid_clock.cc:648] HybridClock initialized: now 1786091471597513 us; error 38 us; skew 500 ppm
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:11.601735 18675 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:31:11.602820 18675 webserver.cc:460] Webserver started at http://127.1.157.65:35107/ using document root <none> and password file <none>
I20260807 08:31:11.603592 18675 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:11.603682 18675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:11.603995 18675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:11.620635 18675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "0fbd519d0ab34118a712673d527a1d4e"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "8ef2c0f38714e3e313462a065ac3f6f7"
server_key_iv: "5de816db9788d8efbf87d3f879f5217f"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:11.621431 18675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "0fbd519d0ab34118a712673d527a1d4e"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "8ef2c0f38714e3e313462a065ac3f6f7"
server_key_iv: "5de816db9788d8efbf87d3f879f5217f"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:11.628206 18675 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 08:31:11.645067 18705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:11.646763 18675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:31:11.646960 18675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "0fbd519d0ab34118a712673d527a1d4e"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "8ef2c0f38714e3e313462a065ac3f6f7"
server_key_iv: "5de816db9788d8efbf87d3f879f5217f"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:11.647171 18675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:11.682510 18675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:11.685357 18675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:11.685523 18675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:11.686089 18675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:11.687168 18675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:11.687219 18675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:11.687269 18675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:11.687310 18675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:11.702997 18675 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:33055
I20260807 08:31:11.703938 18675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:31:11.708326 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 18675
I20260807 08:31:11.708454 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:31:11.708757 1653 external_mini_cluster.cc:1428] Setting key a4d8ead9ad3ec9c9396c002c70e9dcdd
I20260807 08:31:11.714787 18881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:33055 every 8 connection(s)
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:31:11.732159 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:11.716503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39937 (local address 127.1.157.126:36069)
0807 08:31:11.718027 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:11.718032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:11.718497 (+ 465us) server_negotiation.cc:407] Connection header received
0807 08:31:11.719938 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:11.719941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:11.719996 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:11.720060 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:11.721686 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:11.722564 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:11.724269 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:11.724624 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:11.724697 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:11.727971 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:11.727975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:11.727978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:11.728003 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:11.729754 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:11.730362 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:11.730366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:11.730367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:11.730461 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:11.730867 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:11.730870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:11.730872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:11.731099 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:11.731186 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:11.731847 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 08:31:11.731974 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1440,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:31:11.733044 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:11.717019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:36069 (local address 127.1.157.65:39937)
0807 08:31:11.718314 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:11.718406 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:11.719701 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:11.720477 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:11.720487 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:11.720832 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:11.721522 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:11.721542 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:11.722718 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:11.722722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:11.724124 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:11.724135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:11.724943 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:11.724949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:11.725187 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:11.725946 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:11.725966 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:11.727792 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:11.729882 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:11.729888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:11.729902 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:11.730228 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:11.730576 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:11.730579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:11.730581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:11.730751 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:11.731273 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:11.731280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:11.731568 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 08:31:11.731766 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 08:31:11.732058 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":132,"threads_started":1}
I20260807 08:31:11.734539 18882 heartbeater.cc:347] Connected to a master server at 127.1.157.126:36069
I20260807 08:31:11.734773 18882 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:11.735334 18882 heartbeater.cc:511] Master 127.1.157.126:36069 requested a full tablet report, sending...
I20260807 08:31:11.736982 18547 ts_manager.cc:194] Registered new tserver with Master: 0fbd519d0ab34118a712673d527a1d4e (127.1.157.65:33055)
I20260807 08:31:11.738011 18547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:39937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:31:11.753392 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:47383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:31:11.936523 18893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:11.936771 18893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:11.936831 18893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:11.940704 18893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:11.940757 18893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:11.940832 18893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:31:11.946164 18893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.157.84:47383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:11.947577 18893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:11.948899 18893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:11.960619 18905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:11.960645 18909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:11.960776 18893 server_base.cc:1034] running on GCE node
W20260807 08:31:11.960983 18903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:11.961481 18893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:11.962301 18893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:11.963516 18893 hybrid_clock.cc:648] HybridClock initialized: now 1786091471963491 us; error 39 us; skew 500 ppm
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:11.967813 18893 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:31:11.968809 18893 webserver.cc:460] Webserver started at http://127.1.157.66:39423/ using document root <none> and password file <none>
I20260807 08:31:11.969452 18893 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:11.969534 18893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:11.969831 18893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:11 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:11.987571 18893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "e51a688f91df43019341e61d0b8d11e6"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "9807872b362bf84e0b9a5c3f8df52a1b"
server_key_iv: "b50fd94649d5604888c7cb86271e9553"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:11.988361 18893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "e51a688f91df43019341e61d0b8d11e6"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "9807872b362bf84e0b9a5c3f8df52a1b"
server_key_iv: "b50fd94649d5604888c7cb86271e9553"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:11.999078 18893 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.000s
I20260807 08:31:12.020242 18917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.021894 18893 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 08:31:12.022039 18893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "e51a688f91df43019341e61d0b8d11e6"
format_stamp: "Formatted at 2026-08-07 08:31:11 on dist-test-slave-6zbq"
server_key: "9807872b362bf84e0b9a5c3f8df52a1b"
server_key_iv: "b50fd94649d5604888c7cb86271e9553"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:12.022167 18893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:12.044396 18893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:12.047801 18893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:12.048041 18893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:12.048817 18893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:12.049911 18893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:12.050033 18893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.050140 18893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:12.050223 18893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.069526 18893 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:44485
I20260807 08:31:12.070742 18893 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:31:12.074743 19102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:44485 every 8 connection(s)
I20260807 08:31:12.075862 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 18893
I20260807 08:31:12.076002 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:31:12.076366 1653 external_mini_cluster.cc:1428] Setting key b22dad011c01d26421b07615a7df0031
Aug 07 08:31:12 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:12.089495 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.076298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:49007 (local address 127.1.157.126:36069)
0807 08:31:12.076417 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.076421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.076438 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:31:12.077360 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.077363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.077415 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.077464 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:12.079300 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.080120 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.081923 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.082282 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.082388 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.085681 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:12.085685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:12.085688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:12.085718 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:12.087379 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.087981 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.087985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.087987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.088061 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.088381 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.088384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.088386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.088557 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:12.088643 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.089224 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.089344 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:31:12.091362 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.075267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:36069 (local address 127.1.157.66:49007)
0807 08:31:12.075780 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.075832 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.077100 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.077847 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.077856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.078305 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:12.079118 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.079139 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.080286 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.080289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.081780 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.081792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.082589 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.082593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.082842 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.083475 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:12.083495 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:12.085487 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:12.087514 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:12.087519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:12.087533 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:12.087863 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:12.088149 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:12.088151 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:12.088153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:12.088283 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:12.088659 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:12.088665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:12.088949 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.089976 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.090300 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":158,"threads_started":1}
I20260807 08:31:12.093021 19103 heartbeater.cc:347] Connected to a master server at 127.1.157.126:36069
I20260807 08:31:12.093287 19103 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:12.093921 19103 heartbeater.cc:511] Master 127.1.157.126:36069 requested a full tablet report, sending...
I20260807 08:31:12.095155 18547 ts_manager.cc:194] Registered new tserver with Master: e51a688f91df43019341e61d0b8d11e6 (127.1.157.66:44485)
I20260807 08:31:12.095875 18547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:49007
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:31:12.117206 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:47383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:12.282670 19114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:12.283063 19114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:12.283135 19114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:12.289021 19114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:12.289116 19114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:12.289216 19114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:31:12.297243 19114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.157.84:47383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:12.298945 19114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:12.300503 19114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:12.315488 19120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:12.315589 19123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:12.315490 19121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:12.317004 19114 server_base.cc:1034] running on GCE node
I20260807 08:31:12.317535 19114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:12.318428 19114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:12.319634 19114 hybrid_clock.cc:648] HybridClock initialized: now 1786091472319485 us; error 159 us; skew 500 ppm
Aug 07 08:31:12 dist-test-slave-6zbq krb5kdc[17789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091472, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:12.323712 19114 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:31:12.324901 19114 webserver.cc:460] Webserver started at http://127.1.157.67:43929/ using document root <none> and password file <none>
I20260807 08:31:12.325580 19114 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:12.325655 19114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:12.325867 19114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:12 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091472, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:12.345675 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "756c83f04a20493c8cc121c8a2a9a898"
format_stamp: "Formatted at 2026-08-07 08:31:12 on dist-test-slave-6zbq"
server_key: "7713aec52f98166e2b4b258b4689e39b"
server_key_iv: "d89bcdfebbf019f2bda08b34abde0c6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:12.346817 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "756c83f04a20493c8cc121c8a2a9a898"
format_stamp: "Formatted at 2026-08-07 08:31:12 on dist-test-slave-6zbq"
server_key: "7713aec52f98166e2b4b258b4689e39b"
server_key_iv: "d89bcdfebbf019f2bda08b34abde0c6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:12.353176 19114 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260807 08:31:12.384215 19131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.386270 19114 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 08:31:12.386449 19114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "756c83f04a20493c8cc121c8a2a9a898"
format_stamp: "Formatted at 2026-08-07 08:31:12 on dist-test-slave-6zbq"
server_key: "7713aec52f98166e2b4b258b4689e39b"
server_key_iv: "d89bcdfebbf019f2bda08b34abde0c6a"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:12.386600 19114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 08:31:12 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091471, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:12.419046 19114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:12.422992 19114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:12.423252 19114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:12.424000 19114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:12.425165 19114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:12.425271 19114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.425372 19114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:12.425451 19114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:12.446302 19114 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:36079
I20260807 08:31:12.447587 19114 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:31:12.451406 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 19114
I20260807 08:31:12.451540 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:31:12.451874 1653 external_mini_cluster.cc:1428] Setting key 5d3984ef05b23c4401610fa16ca3c9b1
I20260807 08:31:12.463147 19299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:36079 every 8 connection(s)
Aug 07 08:31:12 dist-test-slave-6zbq krb5kdc[17789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091472, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:12.471016 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.456173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:37671 (local address 127.1.157.126:36069)
0807 08:31:12.456297 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.456301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.457454 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 08:31:12.458598 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.458602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.458659 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.458718 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:12.460342 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.461205 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.462860 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.463210 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.463288 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.466814 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:12.466817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:12.466820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:12.466845 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:12.468538 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.469176 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.469180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.469182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.469277 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.469694 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.469697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.469699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.469924 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:12.470010 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.470723 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.470856 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:31:12.472494 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.456766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:36069 (local address 127.1.157.67:37671)
0807 08:31:12.457303 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.457357 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.458327 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.459151 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.459159 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.459493 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:12.460165 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.460189 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.461356 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.461359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.462725 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.462737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.463494 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.463499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.463683 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.464839 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:12.464859 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:12.466626 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:12.468673 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:12.468678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:12.468694 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:12.469037 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:12.469387 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:12.469389 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:12.469391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:12.469571 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:12.470078 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:12.470084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:12.470421 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.471189 (+ 768us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.471508 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":160,"threads_started":1}
I20260807 08:31:12.473553 19300 heartbeater.cc:347] Connected to a master server at 127.1.157.126:36069
I20260807 08:31:12.473804 19300 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:12.474400 19300 heartbeater.cc:511] Master 127.1.157.126:36069 requested a full tablet report, sending...
I20260807 08:31:12.475395 18547 ts_manager.cc:194] Registered new tserver with Master: 756c83f04a20493c8cc121c8a2a9a898 (127.1.157.67:36079)
I20260807 08:31:12.475919 18547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:37671
I20260807 08:31:12.479300 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:31:12.499711 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.481691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.1.157.126:36069)
0807 08:31:12.481800 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.481804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.490674 (+ 8870us) server_negotiation.cc:407] Connection header received
0807 08:31:12.491299 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.491302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.491361 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.491451 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:12.492448 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.493262 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.494452 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.494768 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.494831 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.496489 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:12.496493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:12.496496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:12.496521 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:12.497974 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.498446 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.498450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.498452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.498530 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:12.498826 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:12.498830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:12.498831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:12.499025 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:12.499125 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.499442 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.499551 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:31:12.503023 18547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:31:12.507061 18547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:31:12.524510 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.517249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33055 (local address 127.0.0.1:36758)
0807 08:31:12.518692 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.518707 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.518804 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.519975 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.519978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.519996 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:12.520083 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.520091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.521579 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.521582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.522854 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.522864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.523918 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.523922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.524029 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.524108 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.524220 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.524342 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":20,"thread_start_us":104,"threads_started":1}
I20260807 08:31:12.524534 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.517352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36758 (local address 127.1.157.65:33055)
0807 08:31:12.519636 (+ 2284us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.519641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.519659 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:31:12.519717 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.519722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.519852 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.519974 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:12.520214 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.521405 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.523004 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.523781 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.524100 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.524242 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.524295 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.524390 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2198,"thread_start_us":94,"threads_started":1}
I20260807 08:31:12.525393 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.517011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44485 (local address 127.0.0.1:37816)
0807 08:31:12.518225 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.518277 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.518452 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.519433 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.519439 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.519466 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:12.519603 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.519610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.521107 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.521117 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.522699 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.522711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.524025 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.524029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.525096 (+ 1067us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.525132 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.525204 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.525267 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260807 08:31:12.525897 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.517860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.1.157.66:44485)
0807 08:31:12.519014 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.519026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.519047 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:31:12.519109 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.519113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.519290 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.519412 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:12.519756 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.520955 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.522869 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.523697 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.523803 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.523958 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.525659 (+ 1701us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.525760 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":96,"threads_started":1}
I20260807 08:31:12.526389 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.518096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36079 (local address 127.0.0.1:39558)
0807 08:31:12.520200 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.520212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.520295 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.521499 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.521501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.521519 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:12.521618 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.521624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.522989 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.522992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.524810 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.524820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.525993 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.525997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.526091 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.526112 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.526181 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.526245 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2046,"thread_start_us":87,"threads_started":1}
I20260807 08:31:12.526897 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.518510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39558 (local address 127.1.157.67:36079)
0807 08:31:12.518893 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.518906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.520379 (+ 1473us) server_negotiation.cc:407] Connection header received
0807 08:31:12.520454 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.520459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.520625 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.521358 (+ 733us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:12.521734 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.522866 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.524956 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.525704 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.525801 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.525930 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.526663 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.526764 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":173,"threads_started":1}
I20260807 08:31:12.528040 18779 tablet_service.cc:1467] Processing CreateTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 (DEFAULT_TABLE table=test-table [id=7714416fdd12421e83672acdbd395db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:12.529227 18779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:12.529094 18998 tablet_service.cc:1467] Processing CreateTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 (DEFAULT_TABLE table=test-table [id=7714416fdd12421e83672acdbd395db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:12.529495 19192 tablet_service.cc:1467] Processing CreateTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 (DEFAULT_TABLE table=test-table [id=7714416fdd12421e83672acdbd395db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:12.530049 18998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:12.530448 19192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:12.537904 19323 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Bootstrap starting.
I20260807 08:31:12.538301 19324 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: Bootstrap starting.
I20260807 08:31:12.540860 19324 tablet_bootstrap.cc:654] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:12.543290 19324 log.cc:824] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:12.543519 19323 tablet_bootstrap.cc:654] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:12.544027 19326 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Bootstrap starting.
I20260807 08:31:12.544359 19323 log.cc:824] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:12.546334 19324 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: No bootstrap required, opened a new log
I20260807 08:31:12.546579 19324 ts_tablet_manager.cc:1397] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:31:12.546618 19326 tablet_bootstrap.cc:654] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:12.548636 19326 log.cc:824] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:12.550889 19323 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: No bootstrap required, opened a new log
I20260807 08:31:12.551070 19323 ts_tablet_manager.cc:1397] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260807 08:31:12.551268 19326 tablet_bootstrap.cc:492] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: No bootstrap required, opened a new log
I20260807 08:31:12.551440 19326 ts_tablet_manager.cc:1397] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:31:12.551842 19324 raft_consensus.cc:348] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.552096 19324 raft_consensus.cc:374] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:12.552142 19324 raft_consensus.cc:729] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fbd519d0ab34118a712673d527a1d4e, State: Initialized, Role: FOLLOWER
I20260807 08:31:12.552659 19324 consensus_queue.cc:260] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.553778 19324 ts_tablet_manager.cc:1428] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 08:31:12.554220 18882 heartbeater.cc:503] Master 127.1.157.126:36069 was elected leader, sending a full tablet report...
I20260807 08:31:12.555078 19323 raft_consensus.cc:348] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.555346 19323 raft_consensus.cc:374] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:12.555444 19323 raft_consensus.cc:729] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756c83f04a20493c8cc121c8a2a9a898, State: Initialized, Role: FOLLOWER
I20260807 08:31:12.555935 19323 consensus_queue.cc:260] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.556900 19323 ts_tablet_manager.cc:1428] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:31:12.557279 19300 heartbeater.cc:503] Master 127.1.157.126:36069 was elected leader, sending a full tablet report...
I20260807 08:31:12.559201 19326 raft_consensus.cc:348] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.559489 19326 raft_consensus.cc:374] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:12.559669 19326 raft_consensus.cc:729] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51a688f91df43019341e61d0b8d11e6, State: Initialized, Role: FOLLOWER
I20260807 08:31:12.560159 19326 consensus_queue.cc:260] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.560870 19103 heartbeater.cc:503] Master 127.1.157.126:36069 was elected leader, sending a full tablet report...
I20260807 08:31:12.561065 19326 ts_tablet_manager.cc:1428] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260807 08:31:12.575042 19107 tablet.cc:2369] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:31:12.580000 19343 raft_consensus.cc:482] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:31:12.580225 19343 raft_consensus.cc:504] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.582283 19343 leader_election.cc:288] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fbd519d0ab34118a712673d527a1d4e (127.1.157.65:33055), 756c83f04a20493c8cc121c8a2a9a898 (127.1.157.67:36079)
I20260807 08:31:12.586704 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.581949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33055 (local address 127.1.157.66:52835)
0807 08:31:12.582212 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.582226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.582305 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.582797 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.582801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.582820 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:12.582940 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.582946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.583968 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.583972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.585251 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.585263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.586259 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.586263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.586388 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.586413 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.586490 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.586561 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":26}
I20260807 08:31:12.587352 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.582452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:52835 (local address 127.1.157.65:33055)
0807 08:31:12.582555 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.582560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.582572 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:31:12.582627 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.582629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.582669 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.582732 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:12.583073 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.583831 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.585410 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.586015 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.586143 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.586197 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.587156 (+ 959us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.587233 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:31:12.587666 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.582050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46845 (local address 127.1.157.67:36079)
0807 08:31:12.582172 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.582176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.582736 (+ 560us) server_negotiation.cc:407] Connection header received
0807 08:31:12.582877 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.582880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.582926 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.582971 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:12.583517 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.584274 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.586143 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.586770 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.586857 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.586900 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.587481 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.587554 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:31:12.588033 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.581987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36079 (local address 127.1.157.66:46845)
0807 08:31:12.582590 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.582604 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:12.582750 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:12.583099 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:12.583102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:12.583129 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:12.583384 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.583394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.584505 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.584508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.585994 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:12.586005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.587184 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.587189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:12.587338 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.587361 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:31:12.587794 (+ 433us) client_negotiation.cc:240] Negotiation successful
0807 08:31:12.587868 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":33,"thread_start_us":170,"threads_started":1}
I20260807 08:31:12.588153 18810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737fe0a1d19e4930a3a3b0fde0eb9dd7" candidate_uuid: "e51a688f91df43019341e61d0b8d11e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fbd519d0ab34118a712673d527a1d4e" is_pre_election: true
I20260807 08:31:12.588466 18810 raft_consensus.cc:2454] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51a688f91df43019341e61d0b8d11e6 in term 0.
I20260807 08:31:12.588708 19234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737fe0a1d19e4930a3a3b0fde0eb9dd7" candidate_uuid: "e51a688f91df43019341e61d0b8d11e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756c83f04a20493c8cc121c8a2a9a898" is_pre_election: true
I20260807 08:31:12.589056 19234 raft_consensus.cc:2454] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51a688f91df43019341e61d0b8d11e6 in term 0.
I20260807 08:31:12.589196 18921 leader_election.cc:302] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fbd519d0ab34118a712673d527a1d4e, e51a688f91df43019341e61d0b8d11e6; no voters:
I20260807 08:31:12.589486 19343 raft_consensus.cc:2781] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:31:12.589553 19343 raft_consensus.cc:482] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:31:12.589589 19343 raft_consensus.cc:3037] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:12.591485 19343 raft_consensus.cc:504] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.591885 19343 leader_election.cc:288] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [CANDIDATE]: Term 1 election: Requested vote from peers 0fbd519d0ab34118a712673d527a1d4e (127.1.157.65:33055), 756c83f04a20493c8cc121c8a2a9a898 (127.1.157.67:36079)
I20260807 08:31:12.592314 19234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737fe0a1d19e4930a3a3b0fde0eb9dd7" candidate_uuid: "e51a688f91df43019341e61d0b8d11e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756c83f04a20493c8cc121c8a2a9a898"
I20260807 08:31:12.592379 18810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737fe0a1d19e4930a3a3b0fde0eb9dd7" candidate_uuid: "e51a688f91df43019341e61d0b8d11e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fbd519d0ab34118a712673d527a1d4e"
I20260807 08:31:12.592437 19234 raft_consensus.cc:3037] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:12.592494 18810 raft_consensus.cc:3037] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:12.594072 18810 raft_consensus.cc:2454] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51a688f91df43019341e61d0b8d11e6 in term 1.
I20260807 08:31:12.594146 19234 raft_consensus.cc:2454] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51a688f91df43019341e61d0b8d11e6 in term 1.
I20260807 08:31:12.594535 18921 leader_election.cc:302] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fbd519d0ab34118a712673d527a1d4e, e51a688f91df43019341e61d0b8d11e6; no voters:
I20260807 08:31:12.594857 19343 raft_consensus.cc:2781] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:31:12.595103 19343 raft_consensus.cc:686] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 1 LEADER]: Becoming Leader. State: Replica: e51a688f91df43019341e61d0b8d11e6, State: Running, Role: LEADER
I20260807 08:31:12.595468 19343 consensus_queue.cc:237] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } }
I20260807 08:31:12.599133 18546 catalog_manager.cc:5495] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e51a688f91df43019341e61d0b8d11e6 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "e51a688f91df43019341e61d0b8d11e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51a688f91df43019341e61d0b8d11e6" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 } health_report { overall_health: UNKNOWN } } }
I20260807 08:31:12.624879 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.620314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.1.157.66:44485)
0807 08:31:12.620441 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:12.620446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:12.620463 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:31:12.620644 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:12.620648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:12.620715 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:12.620831 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:31:12.621102 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.621890 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.623201 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:12.623521 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:12.623635 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:12.623934 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:31:12.624476 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:31:12.624600 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:12.624682 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 08:31:12.624742 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:31:12.630623 19234 raft_consensus.cc:1261] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 1 FOLLOWER]: Refusing update from remote peer e51a688f91df43019341e61d0b8d11e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:31:12.631306 19343 consensus_queue.cc:1035] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "756c83f04a20493c8cc121c8a2a9a898" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:31:12.631814 18810 raft_consensus.cc:1261] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 0fbd519d0ab34118a712673d527a1d4e [term 1 FOLLOWER]: Refusing update from remote peer e51a688f91df43019341e61d0b8d11e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:31:12.632380 19343 consensus_queue.cc:1035] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fbd519d0ab34118a712673d527a1d4e" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 33055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:31:12.643787 19350 mvcc.cc:205] Tried to move back new op lower bound from 7315830671888441344 to 7315830671753912320. Current Snapshot: MvccSnapshot[applied={T|T < 7315830671888441344}]
I20260807 08:31:12.648998 19352 mvcc.cc:205] Tried to move back new op lower bound from 7315830671888441344 to 7315830671753912320. Current Snapshot: MvccSnapshot[applied={T|T < 7315830671888441344}]
I20260807 08:31:12.649519 19354 mvcc.cc:205] Tried to move back new op lower bound from 7315830671888441344 to 7315830671753912320. Current Snapshot: MvccSnapshot[applied={T|T < 7315830671888441344 or (T in {7315830671888441344})}]
I20260807 08:31:12.659238 18547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:31:12.659408 18547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:31:12.662211 18547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 885d9e7fd5214081a89ed29c7384473f: Sending DeleteTablet for 3 replicas of tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7
I20260807 08:31:12.662976 18998 tablet_service.cc:1514] Processing DeleteTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:12 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:37816
I20260807 08:31:12.663210 19192 tablet_service.cc:1514] Processing DeleteTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:12 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:39558
I20260807 08:31:12.664130 18779 tablet_service.cc:1514] Processing DeleteTablet for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:12 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:36758
I20260807 08:31:12.664501 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 18675
I20260807 08:31:12.680809 19365 tablet_replica.cc:330] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: stopping tablet replica
I20260807 08:31:12.681334 19365 raft_consensus.cc:2229] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:31:12.681840 19365 raft_consensus.cc:2258] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:31:12.682308 19363 tablet_replica.cc:330] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: stopping tablet replica
I20260807 08:31:12.682672 19363 raft_consensus.cc:2229] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:31:12.683109 19363 raft_consensus.cc:2258] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:31:12.683818 19365 ts_tablet_manager.cc:1905] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:31:12.684604 19363 ts_tablet_manager.cc:1905] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:31:12.687455 19363 ts_tablet_manager.cc:1918] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 08:31:12.687773 18525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:33055 (error 108)
I20260807 08:31:12.688546 19363 log.cc:1196] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P e51a688f91df43019341e61d0b8d11e6: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/737fe0a1d19e4930a3a3b0fde0eb9dd7
I20260807 08:31:12.688812 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 18893
I20260807 08:31:12.694026 19365 ts_tablet_manager.cc:1918] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:31:12.694111 19365 log.cc:1196] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/737fe0a1d19e4930a3a3b0fde0eb9dd7
I20260807 08:31:12.694451 19365 ts_tablet_manager.cc:1939] T 737fe0a1d19e4930a3a3b0fde0eb9dd7 P 756c83f04a20493c8cc121c8a2a9a898: Deleting consensus metadata
I20260807 08:31:12.695880 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.695528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:33055 (local address 127.0.0.1:36766)
0807 08:31:12.695714 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.695783 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 08:31:12.695798 18522 catalog_manager.cc:4841] TS 756c83f04a20493c8cc121c8a2a9a898 (127.1.157.67:36079): tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7 (table test-table [id=7714416fdd12421e83672acdbd395db8]) successfully deleted
W20260807 08:31:12.696223 18525 catalog_manager.cc:4584] TS 0fbd519d0ab34118a712673d527a1d4e (127.1.157.65:33055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 737fe0a1d19e4930a3a3b0fde0eb9dd7: Network error: Client connection negotiation failed: client connection to 127.1.157.65:33055: connect: Connection refused (error 111)
I20260807 08:31:12.700986 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 19114
I20260807 08:31:12.705051 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:12.703006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44485 (local address 127.0.0.1:37840)
0807 08:31:12.704909 (+ 1903us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:12.704967 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.66:44485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1826}
I20260807 08:31:12.715430 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 18492
I20260807 08:31:12.725788 1653 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:31:13.070657 1653 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:31:13.070783 1653 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:31:13.414598 1653 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:31:13.414 UTC [17836] LOG: received smart shutdown request
2026-08-07 08:31:13.418 UTC [17836] LOG: background worker "logical replication launcher" (PID 17851) exited with exit code 1
2026-08-07 08:31:13.418 UTC [17841] LOG: shutting down
2026-08-07 08:31:13.543 UTC [17836] LOG: database system is shut down
2026-08-07T08:31:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45911 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:31:13 dist-test-slave-6zbq krb5kdc[19382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:31:13 dist-test-slave-6zbq krb5kdc[19382](info): set up 2 sockets
Aug 07 08:31:13 dist-test-slave-6zbq krb5kdc[19382](info): commencing operation
krb5kdc: starting...
W20260807 08:31:14.755134 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:31:14 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:31:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:31:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangeradmin_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangeradmin_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangerlookup_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangerlookup_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
I20260807 08:31:14.903064 1653 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasksV4IkV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/postgres -l logfile start
2026-08-07 08:31:16.983 UTC [19432] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 08:31:16.984 UTC [19432] LOG: listening on IPv4 address "127.1.157.84", port 38601
2026-08-07 08:31:16.986 UTC [19432] LOG: listening on Unix socket "/tmp/.s.PGSQL.38601"
2026-08-07 08:31:17.006 UTC [19438] LOG: database system was shut down at 2026-08-07 08:31:15 UTC
2026-08-07 08:31:17.010 UTC [19432] LOG: database system is ready to accept connections
I20260807 08:31:18.045513 1653 mini_postgres.cc:96] Postgres bound to 38601
2026-08-07 08:31:18.052 UTC [19450] FATAL: database "slave" does not exist
127.1.157.84:38601 - accepting connections
I20260807 08:31:18.052897 1653 mini_ranger.cc:161] Starting Ranger...
I20260807 08:31:18.066972 1653 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 08:31:18.413120 1653 mini_ranger.cc:87] Created ranger Postgres database
I20260807 08:31:18.413199 1653 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin
2026-08-07 08:31:18,678 [I] DB FLAVOR :POSTGRES
2026-08-07 08:31:18,679 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:31:18,679 [I] Checking connection..
2026-08-07 08:31:18,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 08:31:18,920 [I] Checking connection passed.
2026-08-07 08:31:18,921 [I] --------- Verifying version history table ---------
2026-08-07 08:31:18,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:31:19,175 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:31:19,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:31:19,408 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 08:31:19,408 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 08:31:19,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 08:31:19.653 UTC [19543] WARNING: there is no transaction in progress
2026-08-07 08:31:19,661 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 08:31:19,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 08:31:19,912 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 08:31:19,912 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:31:19,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 08:31:20,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 08:31:20,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-07 08:31:20,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:31:20,844 [I] Table x_portal_user does not exist in database ranger
2026-08-07 08:31:20,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:31:21,097 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 08:31:21,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:31:21,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:31:21,584 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 08:31:21,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 08:31:22.408 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.423 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.437 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.449 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.462 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.477 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.538 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.546 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.561 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.571 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.581 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.593 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.602 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.608 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.613 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:22.618 UTC [19721] WARNING: there is no transaction in progress
2026-08-07 08:31:23,121 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 08:31:23,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 08:31:23,352 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 08:31:23,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 08:31:23,587 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 08:31:23,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 08:31:23,837 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 08:31:23,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 08:31:24,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 08:31:24,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-07 08:31:24,558 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 08:31:24,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 08:31:24,807 [I] DB_PATCHES have already been applied
I20260807 08:31:24.813014 1653 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasksV4IkV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 08:31:24.813072 1653 mini_ranger.cc:193] Using host: 127.1.157.84
I20260807 08:31:24.817044 1653 mini_ranger.cc:227] Ranger admin URL: http://127.1.157.84:42707
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:31:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 08:31:26.004696 1653 mini_ranger.cc:160] Time spent starting Ranger: real 7.952s user 0.001s sys 0.009s
Aug 07, 2026 8:31:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:31:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:31:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:31:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:31:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 08:31:46.121194 1653 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.157.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.157.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangerkms_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.157.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/rangerkms_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.157.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.157.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.157.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/HTTP_127.1.157.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 08:31:46 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091506, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 08:31:46.204228 1653 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 08:31:46.213711 1653 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 08:31:46.885406 1653 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 08:31:46.885488 1653 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms
I20260807 08:31:46.885506 1653 mini_ranger_kms.cc:227] Using postgres at 127.1.157.84:38601
2026-08-07 08:31:47,051 [I] DB FLAVOR :POSTGRES
2026-08-07 08:31:47,051 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 08:31:47,052 [I] Checking connection
2026-08-07 08:31:47,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 08:31:47,281 [I] connection success
2026-08-07 08:31:47,281 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 08:31:47,281 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 08:31:47,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 08:31:47,506 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 08:31:47,506 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 08:31:47,506 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 08:31:47,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.157.84:38601/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 08:31:47,748 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 08:31:48.174024 1653 mini_ranger_kms.cc:310] Created kms service
I20260807 08:31:48.364260 1653 mini_ranger_kms.cc:326] Created kudu user
I20260807 08:31:48.402953 1653 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 08:31:48 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091506, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:48.753404 1653 mini_ranger_kms.cc:384] Added ranger policy
I20260807 08:31:48.753552 1653 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms:/tmp/dist-test-tasksV4IkV/build/debug/bin/postgresql.jar:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasksV4IkV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 08:31:48.753572 1653 mini_ranger_kms.cc:240] Using host: 127.1.157.84
I20260807 08:31:48.758483 1653 mini_ranger_kms.cc:276] Ranger KMS PID: 20031
I20260807 08:31:48.758543 1653 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.157.84:54401
Aug 07, 2026 8:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:31:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:31:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 08:31:50 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:51.115047 1653 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.911s user 0.011s sys 0.140s
I20260807 08:31:51.115177 1653 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 08:31:51.115263 1653 mini_ranger_kms.cc:401] 127.1.157.84:54401/kms/v1/keys
Aug 07 08:31:51 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
Aug 07 08:31:51 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.157.84@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
Aug 07 08:31:53 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:31:53.978116 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45033
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:45033
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:54.092623 20111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:54.092873 20111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:54.092916 20111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:54.096796 20111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:31:54.096844 20111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:54.096861 20111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:54.096876 20111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:31:54.096890 20111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:31:54.102029 20111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.157.84:54401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:45033
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:54.102962 20111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:54.103863 20111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:54.112969 20119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:54.113047 20118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:54.112946 20121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:54.113314 20111 server_base.cc:1034] running on GCE node
I20260807 08:31:54.113802 20111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:54.114835 20111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:54.116005 20111 hybrid_clock.cc:648] HybridClock initialized: now 1786091514115983 us; error 39 us; skew 500 ppm
Aug 07 08:31:54 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:54.118520 20111 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:31:54.119360 20111 webserver.cc:460] Webserver started at http://127.1.157.126:38413/ using document root <none> and password file <none>
I20260807 08:31:54.119827 20111 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:54.119879 20111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:54.120054 20111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:54 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:54.165704 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "815619e0e8ea4926b2fc41f8cb68ac9b"
format_stamp: "Formatted at 2026-08-07 08:31:54 on dist-test-slave-6zbq"
server_key: "7e94d3c5f36406948bb1310092c65d79"
server_key_iv: "134cc8858612075ba9d02ff87d92b54b"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:54.166474 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "815619e0e8ea4926b2fc41f8cb68ac9b"
format_stamp: "Formatted at 2026-08-07 08:31:54 on dist-test-slave-6zbq"
server_key: "7e94d3c5f36406948bb1310092c65d79"
server_key_iv: "134cc8858612075ba9d02ff87d92b54b"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:54.171074 20111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 08:31:54.186745 20135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:54.188300 20111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 08:31:54.188436 20111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "815619e0e8ea4926b2fc41f8cb68ac9b"
format_stamp: "Formatted at 2026-08-07 08:31:54 on dist-test-slave-6zbq"
server_key: "7e94d3c5f36406948bb1310092c65d79"
server_key_iv: "134cc8858612075ba9d02ff87d92b54b"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:54.188568 20111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:54.213471 20111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:54.216089 20111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:54.216264 20111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:54.223737 20111 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:45033
I20260807 08:31:54.223757 20213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:45033 every 8 connection(s)
I20260807 08:31:54.224686 20111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:31:54.228036 20214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:54.234661 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 20111
I20260807 08:31:54.234771 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:31:54.235061 1653 external_mini_cluster.cc:1428] Setting key 54bef9efd94e2cbea19b1b2ab8ec7753
I20260807 08:31:54.235085 20214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Bootstrap starting.
I20260807 08:31:54.237833 20214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:54.238813 20214 log.cc:824] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:54.240949 20214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: No bootstrap required, opened a new log
Aug 07 08:31:54 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:54.245375 20214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:54.245679 20214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:54.245741 20214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815619e0e8ea4926b2fc41f8cb68ac9b, State: Initialized, Role: FOLLOWER
I20260807 08:31:54.246215 20214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:54.246483 20214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:31:54.246604 20214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:31:54.246766 20214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:54.247694 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:54.236331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.1.157.126:45033)
0807 08:31:54.236715 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:54.236730 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:54.236764 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:31:54.237344 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:54.237360 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:54.237652 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:54.237937 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:54.238724 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:54.239839 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:54.240635 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:54.241138 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:54.241284 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:54.243558 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:54.243563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:54.243576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:54.243600 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:54.245015 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:54.245416 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:54.245421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:54.245425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:54.245525 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:54.245806 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:54.245810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:54.245811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:54.246230 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:54.246454 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:54.246688 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 08:31:54.247047 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260807 08:31:54.247792 20214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:54.248164 20214 leader_election.cc:302] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 815619e0e8ea4926b2fc41f8cb68ac9b; no voters:
I20260807 08:31:54.248520 20214 leader_election.cc:288] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:31:54.248612 20222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:31:54.248831 20222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 LEADER]: Becoming Leader. State: Replica: 815619e0e8ea4926b2fc41f8cb68ac9b, State: Running, Role: LEADER
I20260807 08:31:54.249125 20222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:54.249778 20214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:31:54.250489 20224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } } }
I20260807 08:31:54.250597 20224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:54.250801 20224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 815619e0e8ea4926b2fc41f8cb68ac9b. Latest consensus state: current_term: 1 leader_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } } }
I20260807 08:31:54.250871 20224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:54.251257 20232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:31:54.254585 20232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:31:54.255005 20214 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:31:54.262868 20232 catalog_manager.cc:1347] Generated new cluster ID: de1db859a9db4e9085c13c57208cd690
I20260807 08:31:54.262995 20232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:31:54.276664 20232 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:31:54.277828 20232 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:31:54.290390 20232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Generated new TSK 0
I20260807 08:31:54.291044 20232 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:31:54.949110 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:55.060580 20279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:55.060796 20279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:55.060851 20279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:55.064827 20279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:55.064877 20279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:55.064950 20279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:31:55.070252 20279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:55.071128 20279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:55.072108 20279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:55.081907 20287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.081960 20286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.081961 20290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:55.082181 20279 server_base.cc:1034] running on GCE node
I20260807 08:31:55.082584 20279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:55.083145 20279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:55.084311 20279 hybrid_clock.cc:648] HybridClock initialized: now 1786091515084288 us; error 41 us; skew 500 ppm
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:55.087126 20279 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:31:55.087870 20279 webserver.cc:460] Webserver started at http://127.1.157.65:44861/ using document root <none> and password file <none>
I20260807 08:31:55.088341 20279 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:55.088403 20279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:55.088588 20279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:55.107941 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "645261af1ba049c99a168205c75534c6"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "936d511117dfc2dbf7778eaca16e6667"
server_key_iv: "acb79d41b00bed79772784ead7fd1a21"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.108639 20279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "645261af1ba049c99a168205c75534c6"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "936d511117dfc2dbf7778eaca16e6667"
server_key_iv: "acb79d41b00bed79772784ead7fd1a21"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.113916 20279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 08:31:55.132644 20306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.133884 20279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 08:31:55.133991 20279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "645261af1ba049c99a168205c75534c6"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "936d511117dfc2dbf7778eaca16e6667"
server_key_iv: "acb79d41b00bed79772784ead7fd1a21"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.134091 20279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:55.156608 20279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:55.159216 20279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:55.159371 20279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:55.159924 20279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:55.160730 20279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:55.160785 20279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.160837 20279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:55.160861 20279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.172190 20279 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:43107
I20260807 08:31:55.173300 20279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:31:55.174433 20494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:43107 every 8 connection(s)
I20260807 08:31:55.180716 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 20279
I20260807 08:31:55.180830 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:31:55.181146 1653 external_mini_cluster.cc:1428] Setting key b9477b3b3df5e8f1dd5da4868b444c4d
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:55.194933 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.177903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:41575 (local address 127.1.157.126:45033)
0807 08:31:55.182601 (+ 4698us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:55.182619 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:55.182633 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:31:55.182670 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:55.182673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:55.182718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:55.182803 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:55.184183 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.184924 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.186712 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.187065 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.187227 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.190106 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:55.190110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:55.190113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:55.190139 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:55.192022 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.192606 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.192610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.192612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.192710 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.193116 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.193120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.193121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.193338 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:55.193465 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:55.194597 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0807 08:31:55.194759 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4626,"thread_start_us":110,"threads_started":1}
I20260807 08:31:55.196115 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.176942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.65:41575)
0807 08:31:55.177387 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:55.177437 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:55.178422 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:55.183090 (+ 4668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:55.183099 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:55.183419 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:55.184020 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.184038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.185333 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.185337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.186601 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.186611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.187400 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.187404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.187596 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.188216 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:55.188235 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:55.189920 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:55.192148 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.192154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.192167 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.192489 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.192829 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.192832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.192834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.193003 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.193520 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:55.193525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:55.193846 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 08:31:55.194027 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 08:31:55.194312 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260807 08:31:55.197588 20495 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:55.197824 20495 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:55.198438 20495 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:31:55.200153 20159 ts_manager.cc:194] Registered new tserver with Master: 645261af1ba049c99a168205c75534c6 (127.1.157.65:43107)
I20260807 08:31:55.201660 20159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:41575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:31:55.239296 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 08:31:55.359581 20508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:55.359822 20508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:55.359879 20508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:55.363862 20508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:55.363912 20508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:55.363986 20508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:31:55.368979 20508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:55.369962 20508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:55.371037 20508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:55.379797 20517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.380018 20519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.380267 20516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:55.380313 20508 server_base.cc:1034] running on GCE node
I20260807 08:31:55.380779 20508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:55.381374 20508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:55.382550 20508 hybrid_clock.cc:648] HybridClock initialized: now 1786091515382476 us; error 84 us; skew 500 ppm
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:55.385866 20508 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:31:55.386915 20508 webserver.cc:460] Webserver started at http://127.1.157.66:35835/ using document root <none> and password file <none>
I20260807 08:31:55.387588 20508 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:55.387678 20508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:55.387929 20508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:55.403088 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "51c5856a194344669a38c6e0bbb9d699"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "184d2265b1c4727071dc586981dd9013"
server_key_iv: "d2b876b99479c8dd2c997c89c8c07ec6"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.404265 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "51c5856a194344669a38c6e0bbb9d699"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "184d2265b1c4727071dc586981dd9013"
server_key_iv: "d2b876b99479c8dd2c997c89c8c07ec6"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.414505 20508 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260807 08:31:55.431551 20527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.433172 20508 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 08:31:55.433297 20508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "51c5856a194344669a38c6e0bbb9d699"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "184d2265b1c4727071dc586981dd9013"
server_key_iv: "d2b876b99479c8dd2c997c89c8c07ec6"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.433406 20508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:55.489596 20508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:55.493072 20508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:55.493320 20508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:55.494072 20508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:55.495239 20508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:55.495352 20508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.495450 20508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:55.495514 20508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.508760 20508 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37519
I20260807 08:31:55.509855 20508 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:31:55.510653 20695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37519 every 8 connection(s)
I20260807 08:31:55.517079 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 20508
I20260807 08:31:55.517195 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:31:55.517490 1653 external_mini_cluster.cc:1428] Setting key 3267084f9bee585a5bf67243abf7ba39
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:55.529271 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.512680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:37131 (local address 127.1.157.126:45033)
0807 08:31:55.513972 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:55.513977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:55.513995 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:31:55.514832 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:55.514837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:55.514891 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:55.514942 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:55.516414 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.517232 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.518776 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.519067 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.519141 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.525085 (+ 5944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:55.525088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:55.525091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:55.525117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:55.526938 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.527553 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.527557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.527559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.527650 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.528028 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.528031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.528033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.528254 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:55.528331 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:55.528960 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 08:31:55.529116 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1205}
I20260807 08:31:55.530855 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.513073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.66:37131)
0807 08:31:55.513527 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:55.513577 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:55.514584 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:55.515320 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:55.515329 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:55.515645 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:55.516238 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.516266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.517402 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.517406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.518652 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.518663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.519338 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.519342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.519534 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.520096 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:55.520116 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:55.524904 (+ 4788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:55.527091 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.527097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.527111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.527434 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.527752 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.527755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.527757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.527919 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.528396 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:55.528402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:55.528676 (+ 274us) client_negotiation.cc:769] Sending connection context
0807 08:31:55.529677 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0807 08:31:55.529981 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":120,"threads_started":1}
I20260807 08:31:55.532204 20696 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:55.532423 20696 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:55.532918 20696 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:55.534004 20159 ts_manager.cc:194] Registered new tserver with Master: 51c5856a194344669a38c6e0bbb9d699 (127.1.157.66:37519)
I20260807 08:31:55.534751 20159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:37131
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:31:55.567605 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:55.706871 20703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:55.707091 20703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:55.707146 20703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:55.711177 20703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:55.711231 20703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:55.711304 20703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:31:55.716676 20703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:55.717576 20703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:55.718514 20703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:55.727583 20716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.727914 20715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:55.727923 20719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:55.728137 20703 server_base.cc:1034] running on GCE node
I20260807 08:31:55.728590 20703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:55.729225 20703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:55.732501 20703 hybrid_clock.cc:648] HybridClock initialized: now 1786091515732443 us; error 75 us; skew 500 ppm
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:55.736063 20703 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:31:55.737052 20703 webserver.cc:460] Webserver started at http://127.1.157.67:32881/ using document root <none> and password file <none>
I20260807 08:31:55.737702 20703 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:55.737841 20703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:31:55.738103 20703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:55.763095 20703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "0851bded0a12429eadab4587ee55b7c4"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "a81c20a61cd5d30f4dce17214788b55c"
server_key_iv: "2ba77711e21f446cc7dd9b5a54b782ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.767478 20703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "0851bded0a12429eadab4587ee55b7c4"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "a81c20a61cd5d30f4dce17214788b55c"
server_key_iv: "2ba77711e21f446cc7dd9b5a54b782ad"
server_key_version: "kuduclusterkey@0"
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:55.772969 20703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 08:31:55.803498 20731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.805059 20703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:31:55.805173 20703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "0851bded0a12429eadab4587ee55b7c4"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "a81c20a61cd5d30f4dce17214788b55c"
server_key_iv: "2ba77711e21f446cc7dd9b5a54b782ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:55.805284 20703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:55.847478 20703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:55.851043 20703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:55.851280 20703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:55.851974 20703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:55.853013 20703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:55.853116 20703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.853215 20703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:55.853277 20703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:55.863466 20703 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:40957
I20260807 08:31:55.863657 20926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:40957 every 8 connection(s)
I20260807 08:31:55.864656 20703 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:31:55.870277 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 20703
I20260807 08:31:55.870415 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:31:55.870726 1653 external_mini_cluster.cc:1428] Setting key 82360a8c36fff92567e43d0b6da29f76
Aug 07 08:31:55 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:55.882367 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.866853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:38363 (local address 127.1.157.126:45033)
0807 08:31:55.866987 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:55.866992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:55.868066 (+ 1074us) server_negotiation.cc:407] Connection header received
0807 08:31:55.869297 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:55.869302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:55.869362 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:55.869423 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:55.872509 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.873348 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.875036 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.875391 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.875463 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.878329 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:55.878334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:55.878358 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:55.878386 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:55.879983 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.880567 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.880570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.880572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.880664 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:55.881058 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:55.881062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:55.881064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:55.881314 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:55.881414 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:55.882069 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 08:31:55.882192 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:31:55.883333 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:55.867353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.67:38363)
0807 08:31:55.867904 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:55.867961 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:55.869036 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:55.869841 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:55.869849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:55.871659 (+ 1810us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:55.872331 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.872351 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.873506 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.873509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.874899 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:55.874910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:55.875674 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:55.875677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:55.875868 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:55.876484 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:55.876504 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:55.878133 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:55.880115 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.880122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.880138 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.880460 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.880756 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:55.880759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:55.880761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:55.880953 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:55.881456 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:55.881463 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:55.881778 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 08:31:55.881988 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 08:31:55.882316 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":154,"threads_started":1}
I20260807 08:31:55.884785 20928 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:55.885018 20928 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:55.885532 20928 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:55.886622 20158 ts_manager.cc:194] Registered new tserver with Master: 0851bded0a12429eadab4587ee55b7c4 (127.1.157.67:40957)
I20260807 08:31:55.887372 20158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:38363
I20260807 08:31:55.895982 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:31:55.896721 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 20279
I20260807 08:31:55.911296 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 20508
I20260807 08:31:55.921499 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 20703
I20260807 08:31:55.934158 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 20111
I20260807 08:31:55.943202 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45033
--webserver_interface=127.1.157.126
--webserver_port=38413
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:45033
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:31:56.047825 20933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:56.048033 20933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:56.048069 20933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:56.051894 20933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:31:56.051940 20933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:56.051959 20933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:56.051990 20933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:31:56.052016 20933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:31:56.056980 20933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:45033
--ranger_config_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=38413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:56.057778 20933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:56.058661 20933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:56.065989 20945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.066025 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.065997 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.066246 20933 server_base.cc:1034] running on GCE node
I20260807 08:31:56.066833 20933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:56.067718 20933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:56.068868 20933 hybrid_clock.cc:648] HybridClock initialized: now 1786091516068851 us; error 30 us; skew 500 ppm
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:56.071369 20933 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:31:56.072098 20933 webserver.cc:460] Webserver started at http://127.1.157.126:38413/ using document root <none> and password file <none>
I20260807 08:31:56.072546 20933 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:56.072611 20933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:56.152115 20933 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 08:31:56.154922 20957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.156186 20933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:31:56.156306 20933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "815619e0e8ea4926b2fc41f8cb68ac9b"
format_stamp: "Formatted at 2026-08-07 08:31:54 on dist-test-slave-6zbq"
server_key: "7e94d3c5f36406948bb1310092c65d79"
server_key_iv: "134cc8858612075ba9d02ff87d92b54b"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:56.156829 20933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:56.171607 20933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:56.173893 20933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:56.174052 20933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:56.180670 20933 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:45033
I20260807 08:31:56.180701 21035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:45033 every 8 connection(s)
I20260807 08:31:56.181630 20933 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:31:56.185561 21036 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:31:56.187072 21036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Bootstrap starting.
I20260807 08:31:56.189833 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 20933
I20260807 08:31:56.190212 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:43107
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:45033
--webserver_port=44861
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:31:56.202512 21036 log.cc:824] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:56.208474 21036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:31:56.208786 21036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Bootstrap complete.
I20260807 08:31:56.212577 21036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:56.212939 21036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815619e0e8ea4926b2fc41f8cb68ac9b, State: Initialized, Role: FOLLOWER
I20260807 08:31:56.213369 21036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:56.213482 21036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:31:56.213546 21036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:31:56.213634 21036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:31:56.215412 21036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:56.215692 21036 leader_election.cc:302] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 815619e0e8ea4926b2fc41f8cb68ac9b; no voters:
I20260807 08:31:56.215952 21036 leader_election.cc:288] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:31:56.216054 21041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:31:56.216241 21041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [term 2 LEADER]: Becoming Leader. State: Replica: 815619e0e8ea4926b2fc41f8cb68ac9b, State: Running, Role: LEADER
I20260807 08:31:56.216573 21041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } }
I20260807 08:31:56.217100 21036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:31:56.217423 21041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 815619e0e8ea4926b2fc41f8cb68ac9b. Latest consensus state: current_term: 2 leader_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } } }
I20260807 08:31:56.217532 21041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:56.218138 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815619e0e8ea4926b2fc41f8cb68ac9b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45033 } } }
I20260807 08:31:56.218240 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b [sys.catalog]: This master's current role is: LEADER
I20260807 08:31:56.218694 21046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:31:56.221689 21046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:31:56.222215 21036 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 08:31:56.222285 21046 catalog_manager.cc:1259] Loaded cluster ID: de1db859a9db4e9085c13c57208cd690
I20260807 08:31:56.222457 21046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:31:56.224745 21046 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:31:56.225366 21046 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Loaded TSK: 0
I20260807 08:31:56.226572 21046 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:31:56.327479 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:56.327831 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:56.327907 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:56.333750 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:56.333863 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:56.333977 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:31:56.339469 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:43107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=44861
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:56.340394 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:56.341353 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:56.351534 21092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.351714 21038 server_base.cc:1034] running on GCE node
W20260807 08:31:56.351541 21094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.352134 21089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.352644 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:56.353271 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:56.354403 21038 hybrid_clock.cc:648] HybridClock initialized: now 1786091516354377 us; error 35 us; skew 500 ppm
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:56.357894 21038 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:31:56.359266 21038 webserver.cc:460] Webserver started at http://127.1.157.65:44861/ using document root <none> and password file <none>
I20260807 08:31:56.359997 21038 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:56.360095 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:56.382159 21038 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260807 08:31:56.384667 21103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.386086 21038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:31:56.386206 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "645261af1ba049c99a168205c75534c6"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "936d511117dfc2dbf7778eaca16e6667"
server_key_iv: "acb79d41b00bed79772784ead7fd1a21"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:56.386906 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:56.426620 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:56.429694 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:56.429919 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:56.430619 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:56.431634 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:56.431730 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.431810 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:56.431861 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.441465 21038 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:43107
I20260807 08:31:56.442137 21266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:43107 every 8 connection(s)
I20260807 08:31:56.442631 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:31:56.447427 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 21038
I20260807 08:31:56.447865 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:37519
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:45033
--webserver_port=35835
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:56.463406 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.444960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.65:47167)
0807 08:31:56.445405 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:56.445444 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:56.446336 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:56.450550 (+ 4214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:56.450559 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:56.450823 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:56.451315 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.451329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.452705 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.452709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.453904 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.453916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.455194 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.455199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.455387 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.455949 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:56.455969 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:56.457404 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:56.459744 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.459750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.459764 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.460081 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.460519 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.460522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.460524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.460706 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.461589 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:56.461595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:56.461890 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 08:31:56.462089 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 08:31:56.462418 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":152,"threads_started":1}
I20260807 08:31:56.464898 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.444824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:47167 (local address 127.1.157.126:45033)
0807 08:31:56.445296 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:56.445318 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:56.445551 (+ 233us) server_negotiation.cc:407] Connection header received
0807 08:31:56.446901 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:56.446915 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:56.450025 (+ 3110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:56.451431 (+ 1406us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:56.451677 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.452574 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.454038 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.454817 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.454970 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.457634 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:56.457641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:56.457656 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:56.457690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:56.459596 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.460259 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.460265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.460271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.460408 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.460816 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.460819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.460821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.461405 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:56.461531 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:56.463730 (+ 2199us) server_negotiation.cc:299] Negotiation successful
0807 08:31:56.464009 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":95,"threads_started":1}
I20260807 08:31:56.467672 21268 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:56.467885 21268 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.468323 21268 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
W20260807 08:31:56.470316 21268 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:45033 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:31:56.470464 21268 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.470619 21268 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:56.471493 21268 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.471648 21268 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
W20260807 08:31:56.472426 21268 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:31:56.583372 21274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:56.583585 21274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:56.583649 21274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:56.587570 21274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:56.587625 21274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:56.587698 21274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:31:56.592718 21274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:37519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=35835
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:56.593578 21274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:56.594508 21274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:56.604429 21282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.604638 21274 server_base.cc:1034] running on GCE node
W20260807 08:31:56.606503 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.606582 21285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.606988 21274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:56.607703 21274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:56.608902 21274 hybrid_clock.cc:648] HybridClock initialized: now 1786091516608862 us; error 52 us; skew 500 ppm
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:56.612865 21274 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:31:56.613691 21274 webserver.cc:460] Webserver started at http://127.1.157.66:35835/ using document root <none> and password file <none>
I20260807 08:31:56.614177 21274 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:56.614244 21274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:56.644403 21274 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 08:31:56.648442 21296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.650060 21274 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 08:31:56.650279 21274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "51c5856a194344669a38c6e0bbb9d699"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "184d2265b1c4727071dc586981dd9013"
server_key_iv: "d2b876b99479c8dd2c997c89c8c07ec6"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:56.651051 21274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:56.682947 21274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:56.686559 21274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:56.686782 21274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:56.687481 21274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:56.688479 21274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:56.688581 21274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.688666 21274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:56.688719 21274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.700835 21274 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37519
I20260807 08:31:56.702068 21274 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:31:56.703382 21464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37519 every 8 connection(s)
I20260807 08:31:56.710801 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 21274
I20260807 08:31:56.711293 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.157.84:54401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:40957
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:45033
--webserver_port=32881
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:56.719343 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.705139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58097 (local address 127.1.157.126:45033)
0807 08:31:56.706801 (+ 1662us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:56.706806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:56.706825 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:31:56.707629 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:56.707633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:56.707689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:56.707739 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:56.709195 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.710993 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.712617 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.712968 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.713038 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.715860 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:56.715864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:56.715866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:56.715892 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:56.717294 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.717751 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.717754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.717755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.717828 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.718197 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.718200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.718202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.718408 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:56.718498 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:56.719059 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 08:31:56.719180 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1567}
I20260807 08:31:56.720227 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.705750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.66:58097)
0807 08:31:56.706307 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:56.706399 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:56.707393 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:56.708121 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:56.708129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:56.708436 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:56.709022 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.709039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.711161 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.711165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.712485 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.712495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.713239 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.713242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.713432 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.714003 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:56.714022 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:56.715679 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:56.717403 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.717408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.717421 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.717659 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.717959 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.717962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.717964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.718110 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.718552 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:56.718559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:56.718814 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 08:31:56.718974 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 08:31:56.719306 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":149,"threads_started":1}
I20260807 08:31:56.721508 21466 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:56.721727 21466 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.722229 21466 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
W20260807 08:31:56.723744 21466 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:45033 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:31:56.723853 21466 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.724001 21466 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:56.724711 21466 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.724869 21466 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
W20260807 08:31:56.725582 21466 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:31:56.871727 21472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:31:56.872105 21472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:31:56.872251 21472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:31:56.878238 21472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:31:56.878311 21472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:31:56.878413 21472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:31:56.885891 21472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.157.84:54401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:40957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=32881
--tserver_master_addrs=127.1.157.126:45033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:31:56.887166 21472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:31:56.888479 21472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:31:56.900048 21489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.900146 21479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:31:56.901021 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:31:56.901539 21472 server_base.cc:1034] running on GCE node
I20260807 08:31:56.901960 21472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:31:56.902580 21472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:31:56.903744 21472 hybrid_clock.cc:648] HybridClock initialized: now 1786091516903715 us; error 41 us; skew 500 ppm
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:56.907146 21472 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:31:56.908133 21472 webserver.cc:460] Webserver started at http://127.1.157.67:32881/ using document root <none> and password file <none>
I20260807 08:31:56.908648 21472 fs_manager.cc:359] Metadata directory not provided
I20260807 08:31:56.908728 21472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:56.928464 21472 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.000s
I20260807 08:31:56.935132 21499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.936775 21472 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260807 08:31:56.936901 21472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "0851bded0a12429eadab4587ee55b7c4"
format_stamp: "Formatted at 2026-08-07 08:31:55 on dist-test-slave-6zbq"
server_key: "a81c20a61cd5d30f4dce17214788b55c"
server_key_iv: "2ba77711e21f446cc7dd9b5a54b782ad"
server_key_version: "kuduclusterkey@0"
I20260807 08:31:56.937594 21472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:31:56.950174 21472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:31:56.953648 21472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:31:56.953883 21472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:31:56.954573 21472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:31:56.955597 21472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:31:56.955700 21472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.955868 21472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:31:56.955945 21472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:31:56.967329 21472 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:40957
I20260807 08:31:56.967336 21662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:40957 every 8 connection(s)
I20260807 08:31:56.968596 21472 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:31:56.972890 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 21472
Aug 07 08:31:56 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:31:56.982812 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.973510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.1.157.126:45033)
0807 08:31:56.974015 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:56.974023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:56.974033 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:31:56.974061 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:56.974063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:56.974097 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:56.974169 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:56.974926 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.975440 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.976473 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.976675 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.976754 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.978461 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:56.978465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:56.978468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:56.978495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:56.980251 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.981005 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.981008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.981010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.981166 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.981771 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.981774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.981776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.981959 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:56.982154 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:56.982333 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 08:31:56.982489 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"spinlock_wait_cycles":21888,"thread_start_us":70,"threads_started":1}
I20260807 08:31:56.986433 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.970295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:37515 (local address 127.1.157.126:45033)
0807 08:31:56.970453 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:56.970458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:56.971290 (+ 832us) server_negotiation.cc:407] Connection header received
0807 08:31:56.972167 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:56.972171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:56.972227 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:56.972335 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:56.973326 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.974151 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.975851 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.976185 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.976285 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.978931 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:56.978934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:56.978936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:56.978976 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:56.980443 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.981005 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.981009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.981010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.981138 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:56.981547 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:56.981550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:56.981551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:56.981719 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:56.981807 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:56.986173 (+ 4366us) server_negotiation.cc:299] Negotiation successful
0807 08:31:56.986291 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 08:31:56.987222 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:56.970673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45033 (local address 127.1.157.67:37515)
0807 08:31:56.971118 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:56.971168 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:56.971961 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:56.972461 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:56.972467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:56.972702 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:31:56.973178 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.973195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.974308 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.974312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.975705 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:56.975717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:56.976355 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:56.976360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:56.976544 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:56.977120 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:31:56.977140 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:31:56.978791 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:31:56.980537 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.980543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.980555 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.980881 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.981255 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:31:56.981258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:31:56.981260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:31:56.981439 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:31:56.981864 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:31:56.981870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:31:56.982139 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 08:31:56.982335 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 08:31:56.982630 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260807 08:31:56.988298 21663 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45033
I20260807 08:31:56.988507 21663 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.988919 21663 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
W20260807 08:31:56.990362 21663 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:45033 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 08:31:56.990481 21663 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:56.990630 21663 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:56.991596 20982 ts_manager.cc:194] Registered new tserver with Master: 0851bded0a12429eadab4587ee55b7c4 (127.1.157.67:40957)
I20260807 08:31:56.992760 20982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:37515
Aug 07 08:31:57 dist-test-slave-6zbq krb5kdc[19382](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786091517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:31:57.472653 21268 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:57.472937 21268 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:57.473690 20982 ts_manager.cc:194] Registered new tserver with Master: 645261af1ba049c99a168205c75534c6 (127.1.157.65:43107)
I20260807 08:31:57.474495 20982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:47167
Aug 07 08:31:57 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
I20260807 08:31:57.725800 21466 heartbeater.cc:464] Registering TS with master...
I20260807 08:31:57.726063 21466 heartbeater.cc:511] Master 127.1.157.126:45033 requested a full tablet report, sending...
I20260807 08:31:57.726826 20982 ts_manager.cc:194] Registered new tserver with Master: 51c5856a194344669a38c6e0bbb9d699 (127.1.157.66:37519)
I20260807 08:31:57.727597 20982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:58097
I20260807 08:31:57.736400 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:31:57.748018 21701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.737871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.1.157.126:45033)
0807 08:31:57.738103 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:57.738108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:57.738366 (+ 258us) server_negotiation.cc:407] Connection header received
0807 08:31:57.738505 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:57.738509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:57.738555 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:57.738635 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:31:57.739268 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.739787 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.740556 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.740796 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.740880 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.742218 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:31:57.742221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:31:57.742224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:31:57.742249 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:31:57.746527 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:57.746899 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:57.746902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:57.746903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:57.746980 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:31:57.747238 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:31:57.747241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:31:57.747243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:31:57.747405 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:31:57.747520 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:57.747740 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 08:31:57.747854 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260807 08:31:57.751072 20982 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:31:57.755048 20982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:31:57.775184 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.765546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:43107 (local address 127.0.0.1:44184)
0807 08:31:57.767575 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:57.767619 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:57.767738 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:57.770288 (+ 2550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:57.770294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:57.770318 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:57.770484 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.770491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.771943 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.771946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.773319 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.773330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.774657 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.774661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.774774 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.774812 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:31:57.774911 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:31:57.775019 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1867,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260807 08:31:57.775605 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.765666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.1.157.65:43107)
0807 08:31:57.769744 (+ 4078us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:57.769750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:57.769775 (+ 25us) server_negotiation.cc:407] Connection header received
0807 08:31:57.769850 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:57.769855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:57.770062 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:57.770288 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:57.770619 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.771812 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.773460 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.774291 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.774451 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.774606 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:57.775366 (+ 760us) server_negotiation.cc:299] Negotiation successful
0807 08:31:57.775470 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3971,"thread_start_us":56,"threads_started":1}
I20260807 08:31:57.778666 21173 tablet_service.cc:1467] Processing CreateTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf (DEFAULT_TABLE table=test-table [id=f3604eee241b4dd6a81656f7465d3fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:57.779544 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.765605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37519 (local address 127.0.0.1:48016)
0807 08:31:57.770572 (+ 4967us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:57.770586 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:57.770690 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:57.773357 (+ 2667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:57.773360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:57.773378 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:57.773455 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.773460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.775675 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.775742 (+ 67us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.777903 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.777915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.779168 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.779171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.779266 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.779288 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:31:57.779355 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:31:57.779418 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4904,"thread_start_us":106,"threads_started":1}
I20260807 08:31:57.779807 21173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d52f14d43f14aad8479ef82cbd674bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:57.780019 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.766335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.1.157.66:37519)
0807 08:31:57.767554 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:57.767561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:57.770783 (+ 3222us) server_negotiation.cc:407] Connection header received
0807 08:31:57.770859 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:57.770864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:57.771048 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:57.771186 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:57.773569 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.774728 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.778038 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.778874 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.778973 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.779102 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:57.779778 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 08:31:57.779880 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1066,"thread_start_us":126,"threads_started":1}
I20260807 08:31:57.780066 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.766652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:40957 (local address 127.0.0.1:49978)
0807 08:31:57.771409 (+ 4757us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:57.771422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:57.771507 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:57.772120 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:57.772123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:57.772141 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:57.772238 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.772244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.777401 (+ 5157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.777404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.778503 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:57.778515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.779699 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.779703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:57.779785 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.779807 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:31:57.779876 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:31:57.779938 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4696,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260807 08:31:57.780455 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:57.767032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.1.157.67:40957)
0807 08:31:57.767997 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:57.768003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:57.771595 (+ 3592us) server_negotiation.cc:407] Connection header received
0807 08:31:57.771686 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:57.771691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:57.771859 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:57.771984 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:57.772365 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.777277 (+ 4912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.778644 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:57.779401 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:57.779506 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:57.779633 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:57.780228 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 08:31:57.780325 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":106,"threads_started":1}
I20260807 08:31:57.782819 21370 tablet_service.cc:1467] Processing CreateTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf (DEFAULT_TABLE table=test-table [id=f3604eee241b4dd6a81656f7465d3fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:57.783834 21370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d52f14d43f14aad8479ef82cbd674bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:57.783718 21569 tablet_service.cc:1467] Processing CreateTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf (DEFAULT_TABLE table=test-table [id=f3604eee241b4dd6a81656f7465d3fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:31:57.784734 21569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d52f14d43f14aad8479ef82cbd674bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:31:57.793598 21721 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Bootstrap starting.
I20260807 08:31:57.794651 21719 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Bootstrap starting.
I20260807 08:31:57.796734 21721 tablet_bootstrap.cc:654] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:57.796794 21719 tablet_bootstrap.cc:654] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:57.797653 21719 log.cc:824] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:57.797664 21721 log.cc:824] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:57.799638 21719 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: No bootstrap required, opened a new log
I20260807 08:31:57.799656 21721 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: No bootstrap required, opened a new log
I20260807 08:31:57.799814 21719 ts_tablet_manager.cc:1397] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:31:57.799829 21721 ts_tablet_manager.cc:1397] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:31:57.803692 21719 raft_consensus.cc:348] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.803928 21719 raft_consensus.cc:374] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:57.803969 21719 raft_consensus.cc:729] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c5856a194344669a38c6e0bbb9d699, State: Initialized, Role: FOLLOWER
I20260807 08:31:57.804754 21719 consensus_queue.cc:260] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.804893 21721 raft_consensus.cc:348] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.805197 21721 raft_consensus.cc:374] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:57.805289 21721 raft_consensus.cc:729] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0851bded0a12429eadab4587ee55b7c4, State: Initialized, Role: FOLLOWER
I20260807 08:31:57.805819 21719 ts_tablet_manager.cc:1428] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:31:57.805775 21721 consensus_queue.cc:260] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.806233 21466 heartbeater.cc:503] Master 127.1.157.126:45033 was elected leader, sending a full tablet report...
I20260807 08:31:57.806633 21722 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Bootstrap starting.
I20260807 08:31:57.806732 21721 ts_tablet_manager.cc:1428] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 08:31:57.807024 21663 heartbeater.cc:503] Master 127.1.157.126:45033 was elected leader, sending a full tablet report...
I20260807 08:31:57.811655 21722 tablet_bootstrap.cc:654] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Neither blocks nor log segments found. Creating new log.
I20260807 08:31:57.812310 21722 log.cc:824] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:31:57.814132 21722 tablet_bootstrap.cc:492] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: No bootstrap required, opened a new log
I20260807 08:31:57.814327 21722 ts_tablet_manager.cc:1397] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260807 08:31:57.817404 21722 raft_consensus.cc:348] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.817607 21722 raft_consensus.cc:374] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:31:57.817696 21722 raft_consensus.cc:729] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645261af1ba049c99a168205c75534c6, State: Initialized, Role: FOLLOWER
I20260807 08:31:57.818075 21722 consensus_queue.cc:260] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:57.818917 21722 ts_tablet_manager.cc:1428] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 08:31:57.819239 21268 heartbeater.cc:503] Master 127.1.157.126:45033 was elected leader, sending a full tablet report...
W20260807 08:31:57.945397 21269 tablet.cc:2369] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 08:31:57 dist-test-slave-6zbq krb5kdc[19382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for HTTP/127.1.157.84@KRBTEST.COM
W20260807 08:31:57.955758 21467 tablet.cc:2369] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:31:57.970077 21664 tablet.cc:2369] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:31:58.058077 21726 raft_consensus.cc:482] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:31:58.058323 21726 raft_consensus.cc:504] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:58.059878 21726 leader_election.cc:288] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 645261af1ba049c99a168205c75534c6 (127.1.157.65:43107), 0851bded0a12429eadab4587ee55b7c4 (127.1.157.67:40957)
I20260807 08:31:58.067970 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.061041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46787 (local address 127.1.157.67:40957)
0807 08:31:58.061217 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:58.061222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:58.063175 (+ 1953us) server_negotiation.cc:407] Connection header received
0807 08:31:58.063385 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:58.063389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:58.063439 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:58.063487 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:58.063974 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.064775 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.066631 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.067240 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.067333 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:58.067386 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:58.067756 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 08:31:58.067835 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:31:58.068557 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.060467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:40957 (local address 127.1.157.66:46787)
0807 08:31:58.063088 (+ 2621us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:58.063104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:58.063258 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:58.063639 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:58.063641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:58.063675 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:58.063829 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:58.063836 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.064926 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.064929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:58.066491 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:58.066503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.067484 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.067487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:58.067592 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:58.067618 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:31:58.068335 (+ 717us) client_negotiation.cc:240] Negotiation successful
0807 08:31:58.068415 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2547,"thread_start_us":123,"threads_started":1}
I20260807 08:31:58.069289 21593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d52f14d43f14aad8479ef82cbd674bf" candidate_uuid: "51c5856a194344669a38c6e0bbb9d699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0851bded0a12429eadab4587ee55b7c4" is_pre_election: true
I20260807 08:31:58.069595 21593 raft_consensus.cc:2454] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c5856a194344669a38c6e0bbb9d699 in term 0.
I20260807 08:31:58.070179 21298 leader_election.cc:302] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0851bded0a12429eadab4587ee55b7c4, 51c5856a194344669a38c6e0bbb9d699; no voters:
I20260807 08:31:58.070498 21726 raft_consensus.cc:2781] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:31:58.070572 21726 raft_consensus.cc:482] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:31:58.070609 21726 raft_consensus.cc:3037] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:58.071869 21726 raft_consensus.cc:504] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:58.072209 21726 leader_election.cc:288] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [CANDIDATE]: Term 1 election: Requested vote from peers 645261af1ba049c99a168205c75534c6 (127.1.157.65:43107), 0851bded0a12429eadab4587ee55b7c4 (127.1.157.67:40957)
I20260807 08:31:58.072790 21593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d52f14d43f14aad8479ef82cbd674bf" candidate_uuid: "51c5856a194344669a38c6e0bbb9d699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0851bded0a12429eadab4587ee55b7c4"
I20260807 08:31:58.072909 21593 raft_consensus.cc:3037] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:58.074070 21593 raft_consensus.cc:2454] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c5856a194344669a38c6e0bbb9d699 in term 1.
I20260807 08:31:58.074477 21298 leader_election.cc:302] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0851bded0a12429eadab4587ee55b7c4, 51c5856a194344669a38c6e0bbb9d699; no voters:
I20260807 08:31:58.074697 21726 raft_consensus.cc:2781] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:31:58.074894 21726 raft_consensus.cc:686] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 1 LEADER]: Becoming Leader. State: Replica: 51c5856a194344669a38c6e0bbb9d699, State: Running, Role: LEADER
I20260807 08:31:58.075233 21726 consensus_queue.cc:237] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } }
I20260807 08:31:58.078586 20982 catalog_manager.cc:5495] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51c5856a194344669a38c6e0bbb9d699 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "51c5856a194344669a38c6e0bbb9d699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51c5856a194344669a38c6e0bbb9d699" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 } health_report { overall_health: UNKNOWN } } }
I20260807 08:31:58.079566 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.062488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:33657 (local address 127.1.157.65:43107)
0807 08:31:58.062606 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:58.062610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:58.075716 (+ 13106us) server_negotiation.cc:407] Connection header received
0807 08:31:58.075907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:58.075911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:58.075964 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:58.076012 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:31:58.076403 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.077068 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.078362 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.078908 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.078998 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:58.079047 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:58.079381 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 08:31:58.079455 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:31:58.079900 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.062067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:43107 (local address 127.1.157.66:33657)
0807 08:31:58.075616 (+ 13549us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:58.075630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:31:58.075789 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:31:58.076146 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:31:58.076160 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:31:58.076177 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:31:58.076280 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:58.076285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.077214 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.077217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:58.078209 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:31:58.078220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.079137 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.079140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:31:58.079239 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:58.079262 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:31:58.079694 (+ 432us) client_negotiation.cc:240] Negotiation successful
0807 08:31:58.079769 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13486,"thread_start_us":96,"threads_started":1}
I20260807 08:31:58.081573 21206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d52f14d43f14aad8479ef82cbd674bf" candidate_uuid: "51c5856a194344669a38c6e0bbb9d699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645261af1ba049c99a168205c75534c6" is_pre_election: true
I20260807 08:31:58.081883 21206 raft_consensus.cc:2454] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51c5856a194344669a38c6e0bbb9d699 in term 0.
I20260807 08:31:58.082193 21205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d52f14d43f14aad8479ef82cbd674bf" candidate_uuid: "51c5856a194344669a38c6e0bbb9d699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645261af1ba049c99a168205c75534c6"
I20260807 08:31:58.082298 21205 raft_consensus.cc:3037] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:31:58.083544 21205 raft_consensus.cc:2454] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51c5856a194344669a38c6e0bbb9d699 in term 1.
I20260807 08:31:58.146304 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.141388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48032 (local address 127.1.157.66:37519)
0807 08:31:58.141529 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:31:58.141533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:31:58.141548 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:31:58.141635 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:31:58.141638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:31:58.141703 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:31:58.141822 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:31:58.142195 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.142997 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.144185 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:31:58.144511 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:31:58.144604 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:31:58.145013 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:31:58.145545 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:31:58.145655 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:31:58.146058 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 08:31:58.146119 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:31:58.153093 21593 raft_consensus.cc:1261] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 1 FOLLOWER]: Refusing update from remote peer 51c5856a194344669a38c6e0bbb9d699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:31:58.153899 21738 consensus_queue.cc:1035] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0851bded0a12429eadab4587ee55b7c4" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:31:58.160384 21205 raft_consensus.cc:1261] T 8d52f14d43f14aad8479ef82cbd674bf P 645261af1ba049c99a168205c75534c6 [term 1 FOLLOWER]: Refusing update from remote peer 51c5856a194344669a38c6e0bbb9d699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:31:58.161070 21738 consensus_queue.cc:1035] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "645261af1ba049c99a168205c75534c6" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 43107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:31:58.176358 21741 mvcc.cc:205] Tried to move back new op lower bound from 7315830858347978752 to 7315830858039083008. Current Snapshot: MvccSnapshot[applied={T|T < 7315830858347978752}]
I20260807 08:31:58.176707 21743 mvcc.cc:205] Tried to move back new op lower bound from 7315830858347978752 to 7315830858039083008. Current Snapshot: MvccSnapshot[applied={T|T < 7315830858347978752}]
I20260807 08:31:58.179246 21745 mvcc.cc:205] Tried to move back new op lower bound from 7315830858347978752 to 7315830858039083008. Current Snapshot: MvccSnapshot[applied={T|T < 7315830858347978752}]
I20260807 08:31:58.188884 20982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:31:58.189047 20982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:31:58.191748 20982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 815619e0e8ea4926b2fc41f8cb68ac9b: Sending DeleteTablet for 3 replicas of tablet 8d52f14d43f14aad8479ef82cbd674bf
I20260807 08:31:58.192803 21370 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:58 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:48016
I20260807 08:31:58.193257 21569 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:58 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:49978
I20260807 08:31:58.194126 21173 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d52f14d43f14aad8479ef82cbd674bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:31:58 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:44184
I20260807 08:31:58.195354 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 21038
I20260807 08:31:58.199472 21756 tablet_replica.cc:330] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: stopping tablet replica
I20260807 08:31:58.199779 21756 raft_consensus.cc:2229] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:31:58.200067 21756 raft_consensus.cc:2258] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:31:58.200465 21755 tablet_replica.cc:330] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: stopping tablet replica
I20260807 08:31:58.201783 21755 raft_consensus.cc:2229] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:31:58.202195 21755 raft_consensus.cc:2258] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:31:58.204411 21756 ts_tablet_manager.cc:1905] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:31:58.207015 21755 ts_tablet_manager.cc:1905] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:31:58.211450 21756 ts_tablet_manager.cc:1918] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:31:58.218473 21756 log.cc:1196] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/8d52f14d43f14aad8479ef82cbd674bf
I20260807 08:31:58.218948 21756 ts_tablet_manager.cc:1939] T 8d52f14d43f14aad8479ef82cbd674bf P 0851bded0a12429eadab4587ee55b7c4: Deleting consensus metadata
I20260807 08:31:58.220379 20959 catalog_manager.cc:4841] TS 0851bded0a12429eadab4587ee55b7c4 (127.1.157.67:40957): tablet 8d52f14d43f14aad8479ef82cbd674bf (table test-table [id=f3604eee241b4dd6a81656f7465d3fa3]) successfully deleted
I20260807 08:31:58.221122 21755 ts_tablet_manager.cc:1918] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:31:58.221306 21755 log.cc:1196] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/8d52f14d43f14aad8479ef82cbd674bf
I20260807 08:31:58.221678 21755 ts_tablet_manager.cc:1939] T 8d52f14d43f14aad8479ef82cbd674bf P 51c5856a194344669a38c6e0bbb9d699: Deleting consensus metadata
I20260807 08:31:58.222832 20959 catalog_manager.cc:4841] TS 51c5856a194344669a38c6e0bbb9d699 (127.1.157.66:37519): tablet 8d52f14d43f14aad8479ef82cbd674bf (table test-table [id=f3604eee241b4dd6a81656f7465d3fa3]) successfully deleted
W20260807 08:31:58.242569 20962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:43107 (error 108)
I20260807 08:31:58.243721 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 21274
I20260807 08:31:58.244369 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:31:58.244050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:43107 (local address 127.0.0.1:44186)
0807 08:31:58.244221 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:31:58.244285 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:43107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 08:31:58.244556 20962 catalog_manager.cc:4584] TS 645261af1ba049c99a168205c75534c6 (127.1.157.65:43107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d52f14d43f14aad8479ef82cbd674bf: Network error: Client connection negotiation failed: client connection to 127.1.157.65:43107: connect: Connection refused (error 111)
I20260807 08:31:58.255555 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 21472
I20260807 08:31:58.264763 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 20933
I20260807 08:31:58.273969 1653 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 08:31:58.617524 1653 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 08:31:58.617623 1653 mini_ranger.cc:66] Stopping Ranger...
I20260807 08:31:58.961249 1653 mini_ranger.cc:68] Stopped Ranger
2026-08-07 08:31:58.961 UTC [19432] LOG: received smart shutdown request
2026-08-07 08:31:58.964 UTC [19432] LOG: background worker "logical replication launcher" (PID 19448) exited with exit code 1
2026-08-07 08:31:58.965 UTC [19439] LOG: shutting down
2026-08-07 08:31:59.080 UTC [19432] LOG: database system is shut down
2026-08-07T08:31:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45534 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:31:59 dist-test-slave-6zbq krb5kdc[21772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:31:59 dist-test-slave-6zbq krb5kdc[21772](info): set up 2 sockets
Aug 07 08:31:59 dist-test-slave-6zbq krb5kdc[21772](info): commencing operation
krb5kdc: starting...
W20260807 08:32:00.289269 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:32:00.396813 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43879
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:00.500833 21790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:00.501052 21790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:00.501112 21790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:00.505044 21790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:00.505090 21790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:00.505110 21790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:00.505139 21790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:00.505167 21790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:00.510468 21790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:00.511350 21790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:00.512236 21790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:00.520120 21802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:00.520098 21804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.520130 21790 server_base.cc:1034] running on GCE node
W20260807 08:32:00.520092 21801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.520812 21790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:00.521822 21790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:00.522974 21790 hybrid_clock.cc:648] HybridClock initialized: now 1786091520522959 us; error 27 us; skew 500 ppm
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:00.525417 21790 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:00.526213 21790 webserver.cc:460] Webserver started at http://127.1.157.126:37103/ using document root <none> and password file <none>
I20260807 08:32:00.526732 21790 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:00.526793 21790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:00.526978 21790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:00.528772 21790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "6b6ba67e6b5046b2b605a73f25f32bf4"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "c94601c4a27ca3e23882558c26d098ee"
server_key_iv: "d9848e574d7ce0d4c952e87f574f14a0"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.529275 21790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "6b6ba67e6b5046b2b605a73f25f32bf4"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "c94601c4a27ca3e23882558c26d098ee"
server_key_iv: "d9848e574d7ce0d4c952e87f574f14a0"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.532613 21790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:32:00.534715 21813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:00.535753 21790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:00.535862 21790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "6b6ba67e6b5046b2b605a73f25f32bf4"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "c94601c4a27ca3e23882558c26d098ee"
server_key_iv: "d9848e574d7ce0d4c952e87f574f14a0"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.535948 21790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:00.573259 21790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:00.575675 21790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:00.575831 21790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:00.582273 21790 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43879
I20260807 08:32:00.582360 21896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43879 every 8 connection(s)
I20260807 08:32:00.583119 21790 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:00.585546 21898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:00.588891 21898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:32:00.592190 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 21790
I20260807 08:32:00.592336 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:32:00.592625 1653 external_mini_cluster.cc:1428] Setting key e36c2bee885689c812a87fa60cfab2c4
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.125@KRBTEST.COM; defaulting to no policy
I20260807 08:32:00.602279 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.590776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43879 (local address 127.0.0.1:45058)
0807 08:32:00.591232 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.591276 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.592059 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.592789 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.592793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.593101 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.593327 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.593340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.594795 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.594798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.595520 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.595530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.596111 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.596115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.596236 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.596706 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.596723 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.598209 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.599845 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.599853 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.599862 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.600063 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.600349 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.600352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.600354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.600455 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.601067 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.601072 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.601194 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 08:32:00.601325 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 08:32:00.601594 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"spinlock_wait_cycles":1408,"thread_start_us":133,"threads_started":1}
I20260807 08:32:00.602283 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.590775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.1.157.126:43879)
0807 08:32:00.591119 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.591132 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.591366 (+ 234us) server_negotiation.cc:407] Connection header received
0807 08:32:00.592354 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.592359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.592575 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.592760 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.593474 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.594676 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.595629 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.595979 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.596139 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.598356 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.598364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.598375 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.598398 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.599748 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.600159 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.600164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.600165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.600267 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.600569 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.600573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.600574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.600972 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.601093 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.601424 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.601674 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.125@KRBTEST.COM" created.
I20260807 08:32:00.606541 21898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:32:00.607290 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.606975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37782)
0807 08:32:00.607154 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.607227 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260807 08:32:00.607478 21815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
I20260807 08:32:00.609704 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.609345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37794)
0807 08:32:00.609536 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.609614 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 08:32:00.610333 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:37699: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
I20260807 08:32:00.618440 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:37699
--webserver_interface=127.1.157.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:00.659627 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } attempt: 1
I20260807 08:32:00.660337 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.660071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37802)
0807 08:32:00.660219 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.660271 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 08:32:00.661007 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.660805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37808)
0807 08:32:00.660906 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.660952 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:32:00.661142 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:37699: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
W20260807 08:32:00.724063 21909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:00.724277 21909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:00.724321 21909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:00.728197 21909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:00.728245 21909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:00.728276 21909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:00.728305 21909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:00.728335 21909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:00.729278 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } attempt: 2
I20260807 08:32:00.729988 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.729724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37818)
0807 08:32:00.729857 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.729916 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:32:00.730629 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.730440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37826)
0807 08:32:00.730530 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.730574 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:32:00.730749 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:37699: Network error: Client connection negotiation failed: client connection to 127.1.157.125:37699: connect: Connection refused (error 111)
I20260807 08:32:00.733639 21909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.157.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:00.734509 21909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:00.735361 21909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:00.742938 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:00.742952 21920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.743067 21909 server_base.cc:1034] running on GCE node
W20260807 08:32:00.742951 21924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.743736 21909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:00.744673 21909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:00.745828 21909 hybrid_clock.cc:648] HybridClock initialized: now 1786091520745814 us; error 27 us; skew 500 ppm
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:00.748397 21909 init.cc:377] Logged in from keytab as kudu/127.1.157.125@KRBTEST.COM (short username kudu)
I20260807 08:32:00.749181 21909 webserver.cc:460] Webserver started at http://127.1.157.125:46853/ using document root <none> and password file <none>
I20260807 08:32:00.749650 21909 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:00.749711 21909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:00.749893 21909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:00.751753 21909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data/instance:
uuid: "ca953f0209674744b8f5eedff63fc362"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "054c41c637dd824cf445036f6261821a"
server_key_iv: "43b16247c80c8fc7ee211d27419fbe70"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.752275 21909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal/instance:
uuid: "ca953f0209674744b8f5eedff63fc362"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "054c41c637dd824cf445036f6261821a"
server_key_iv: "43b16247c80c8fc7ee211d27419fbe70"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.755622 21909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:32:00.757678 21936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:00.758735 21909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:00.758848 21909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "ca953f0209674744b8f5eedff63fc362"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "054c41c637dd824cf445036f6261821a"
server_key_iv: "43b16247c80c8fc7ee211d27419fbe70"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.758937 21909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:00.769902 21909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:00.772215 21909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:00.772361 21909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:00.778450 21909 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.125:37699
I20260807 08:32:00.778483 22020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.125:37699 every 8 connection(s)
I20260807 08:32:00.779270 21909 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
I20260807 08:32:00.781697 22021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:00.783540 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 21909
I20260807 08:32:00.783653 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-1/wal/instance
I20260807 08:32:00.783890 1653 external_mini_cluster.cc:1428] Setting key 2f666bec1df7a866de6f2945484ba830
I20260807 08:32:00.785128 22021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.157.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.124@KRBTEST.COM" created.
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:00.799567 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.786781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.1.157.126:43879)
0807 08:32:00.786887 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.786890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.787760 (+ 870us) server_negotiation.cc:407] Connection header received
0807 08:32:00.788802 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.788805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.788854 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.788923 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.790376 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.790898 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.792486 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.792724 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.792801 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.796126 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.796130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.796133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.796162 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.797570 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.798085 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.798089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.798090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.798179 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.798512 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.798515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.798517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.798741 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.798839 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.799275 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.799404 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:32:00.800174 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.787151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43879 (local address 127.0.0.1:45064)
0807 08:32:00.787595 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.787645 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.788584 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.789110 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.789119 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.789553 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.790185 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.790204 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.791025 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.791030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.792360 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.792373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.792869 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.792874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.793065 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.793528 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.793543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.795940 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.797691 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.797696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.797709 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.797984 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.798266 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.798268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.798269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.798429 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.798828 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.798832 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.799035 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 08:32:00.799175 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 08:32:00.799432 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
I20260807 08:32:00.801435 22021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
I20260807 08:32:00.809445 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.801855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37840)
0807 08:32:00.801959 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.801972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.802041 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.802663 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.802666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.802785 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.802866 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.802871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.803995 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.803998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.804509 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.804517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.805068 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.805071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.805166 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.805506 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.805516 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.806670 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.808122 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.808125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.808127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.808302 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.808547 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.808550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.808552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.808622 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.809003 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.809006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.809086 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 08:32:00.809172 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 08:32:00.809296 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 08:32:00.809546 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.801989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.1.157.125:37699)
0807 08:32:00.802259 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.802270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.802293 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:32:00.802335 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.802359 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.802544 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.802684 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.802962 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.803898 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.804607 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.804962 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.805070 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.806787 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.806797 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.806806 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.806825 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.808028 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.808392 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.808397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.808399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.808469 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.808694 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.808697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.808703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.808910 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.809018 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.809255 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.809367 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":96,"threads_started":1}
I20260807 08:32:00.809930 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:33557
--webserver_interface=127.1.157.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:00.811010 22021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:32:00.811578 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.811368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36022)
0807 08:32:00.811462 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.811527 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:32:00.811777 21943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
I20260807 08:32:00.812528 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.812348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36034)
0807 08:32:00.812435 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.812479 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:32:00.812639 22021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
I20260807 08:32:00.821894 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } attempt: 3
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:32:00.829977 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.822335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.1.157.125:37699)
0807 08:32:00.822455 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.822458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.822469 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:32:00.822580 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.822582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.822613 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.822665 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.823484 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.824245 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.825331 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.825542 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.825604 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.827519 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.827522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.827524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.827544 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.828673 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.829093 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.829095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.829096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.829154 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.829377 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.829379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.829380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.829511 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.829575 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.829812 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.829887 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:32:00.830049 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.822261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37842)
0807 08:32:00.822409 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.822423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.822498 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.822694 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.822697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.822869 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.823393 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.823401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.824352 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.824355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.825235 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.825246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.825623 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.825626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.825724 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.826088 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.826101 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.827402 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.828781 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.828784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.828786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.829013 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.829218 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.829221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.829223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.829306 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.829583 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.829586 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.829681 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:32:00.829781 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 08:32:00.829901 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:32:00.830687 21898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:32:00.831288 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.831068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36046)
0807 08:32:00.831176 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.831227 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 08:32:00.831921 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.831688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36056)
0807 08:32:00.831820 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.831869 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 08:32:00.832028 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
I20260807 08:32:00.861856 22021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 1
I20260807 08:32:00.862481 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.862233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36058)
0807 08:32:00.862372 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.862420 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 08:32:00.863101 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.862932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36068)
0807 08:32:00.863013 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.863054 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 08:32:00.863198 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 1
W20260807 08:32:00.863256 22021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
I20260807 08:32:00.863715 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.863493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36070)
0807 08:32:00.863601 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.863653 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 08:32:00.864262 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.864068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36072)
0807 08:32:00.864154 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.864203 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 08:32:00.864393 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
W20260807 08:32:00.918023 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:00.918226 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:00.918262 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:00.922046 22030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:00.922087 22030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:00.922106 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:00.922119 22030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:00.922132 22030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:00.926994 22030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43879,127.1.157.125:37699,127.1.157.124:33557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:33557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.157.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:00.927796 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:00.928601 22030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:32:00.931386 22021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 2
I20260807 08:32:00.932032 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.931784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36078)
0807 08:32:00.931910 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.931963 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:32:00.932809 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.932570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36092)
0807 08:32:00.932683 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.932747 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 08:32:00.932997 22021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
W20260807 08:32:00.936229 22039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:00.936229 22042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.936267 22030 server_base.cc:1034] running on GCE node
W20260807 08:32:00.936401 22038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:00.936913 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:00.937852 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:00.939005 22030 hybrid_clock.cc:648] HybridClock initialized: now 1786091520938992 us; error 26 us; skew 500 ppm
I20260807 08:32:00.939543 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 2
I20260807 08:32:00.940129 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.939902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36106)
0807 08:32:00.940010 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.940061 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:32:00.940766 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.940552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36112)
0807 08:32:00.940652 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.940703 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 08:32:00.940881 21898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:33557: Network error: Client connection negotiation failed: client connection to 127.1.157.124:33557: connect: Connection refused (error 111)
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:00.941828 22030 init.cc:377] Logged in from keytab as kudu/127.1.157.124@KRBTEST.COM (short username kudu)
I20260807 08:32:00.942610 22030 webserver.cc:460] Webserver started at http://127.1.157.124:44477/ using document root <none> and password file <none>
I20260807 08:32:00.943081 22030 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:00.943142 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:00.943315 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:00.945122 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data/instance:
uuid: "974322caf84541489bf03fddd615a269"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "05cf4d7e5fe1310851e968b7c1e3bc83"
server_key_iv: "aa0a8c77a5267670a4e5cf14bdfc4a82"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.945613 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal/instance:
uuid: "974322caf84541489bf03fddd615a269"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "05cf4d7e5fe1310851e968b7c1e3bc83"
server_key_iv: "aa0a8c77a5267670a4e5cf14bdfc4a82"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.948927 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 08:32:00.951030 22056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:00.952071 22030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:32:00.952178 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "974322caf84541489bf03fddd615a269"
format_stamp: "Formatted at 2026-08-07 08:32:00 on dist-test-slave-6zbq"
server_key: "05cf4d7e5fe1310851e968b7c1e3bc83"
server_key_iv: "aa0a8c77a5267670a4e5cf14bdfc4a82"
server_key_version: "encryptionkey@0"
I20260807 08:32:00.952263 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:00.965287 22030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:00.968003 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:00.968158 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:00.974404 22030 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.124:33557
I20260807 08:32:00.974407 22134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.124:33557 every 8 connection(s)
I20260807 08:32:00.975198 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
I20260807 08:32:00.975579 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 22030
I20260807 08:32:00.975723 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786091364559477-1653-0/minicluster-data/master-2/wal/instance
I20260807 08:32:00.975977 1653 external_mini_cluster.cc:1428] Setting key 2fe5675475cb1b227bc3429debc996a9
I20260807 08:32:00.978744 22135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:00.982750 22135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:32:00.988662 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.977071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.1.157.124:33557)
0807 08:32:00.977419 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.977432 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.977464 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:32:00.978025 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.978039 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.978334 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.978617 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.979450 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.980382 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.981523 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.981940 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.982043 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.984527 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.984533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.984547 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.984578 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.986202 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.986621 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.986625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.986629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.986732 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.986973 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.986976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.986978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.987347 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.987532 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.987789 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.988049 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":234,"thread_start_us":107,"threads_started":1}
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:00.992146 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.984345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43879 (local address 127.0.0.1:45070)
0807 08:32:00.984677 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.984721 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.984868 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.985135 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.985139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.985453 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.985553 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.985559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.986232 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.986235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.986838 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.986848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.987255 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.987258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.987398 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.987827 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.987838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.989082 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.990767 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.990776 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.990782 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.990983 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.991232 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.991235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.991237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.991330 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.991679 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.991685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.991787 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 08:32:00.991888 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 08:32:00.992001 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":36,"thread_start_us":125,"threads_started":1}
I20260807 08:32:00.992216 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.984428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.1.157.126:43879)
0807 08:32:00.984547 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.984551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.984789 (+ 238us) server_negotiation.cc:407] Connection header received
0807 08:32:00.984991 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.984994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.985038 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.985106 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.985677 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.986134 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.986949 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.987162 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.987247 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.989207 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.989211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.989214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.989239 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.990673 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.991050 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.991053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.991055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.991139 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.991412 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.991414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.991415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.991578 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.991650 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.991969 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.992075 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:32:00.993058 22135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:32:00.998524 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.990615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.1.157.126:43879)
0807 08:32:00.990922 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.990932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.990944 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:32:00.990999 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.991002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.991040 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.991115 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.991692 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.992372 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.993311 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.993568 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.993656 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.995672 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.995676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.995677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.995698 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.997020 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.997422 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.997425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.997426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.997512 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.997751 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.997754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.997756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.997912 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.998074 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:00.998240 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:32:00.998373 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":92,"threads_started":1}
I20260807 08:32:01.000392 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.993519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.1.157.125:37699)
0807 08:32:00.993646 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.993650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.993664 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:32:00.993706 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.993709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.993750 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.993813 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:00.994166 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.994698 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.995388 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.995601 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.995683 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.997425 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:00.997429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:00.997432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:00.997456 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:00.998845 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.999247 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.999250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.999252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.999332 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:00.999580 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:00.999584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:00.999585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:00.999753 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:00.999834 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.000119 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.000238 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:32:01.000779 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.993423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:37699 (local address 127.0.0.1:37846)
0807 08:32:00.993554 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:00.993566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:00.993629 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:00.993833 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:00.993836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:00.993978 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:00.994058 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.994062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.994786 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.994789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.995291 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:00.995301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:00.995690 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:00.995693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:00.995811 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:00.996254 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:00.996263 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:00.997295 (+ 1032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:00.998937 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.998940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.998942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.999158 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.999419 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:00.999422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:00.999424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:00.999502 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:00.999839 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:00.999842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:00.999924 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:32:01.000565 (+ 641us) client_negotiation.cc:240] Negotiation successful
0807 08:32:01.000684 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 08:32:01.001384 22135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
Aug 07 08:32:00 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:32:01.007505 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:00.999567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.1.157.125:37699)
0807 08:32:00.999829 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:00.999835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:00.999845 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:32:00.999873 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:00.999875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:00.999906 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:00.999970 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.000607 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.001208 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.002068 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.002410 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.002506 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.004727 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.004730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.004731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.004752 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.006024 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.006418 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.006421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.006423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.006506 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.006747 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.006750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.006751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.006944 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.007134 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.007244 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.007344 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
I20260807 08:32:01.009063 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.001760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36138)
0807 08:32:01.001858 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:01.001871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:01.001936 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:01.002115 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:01.002116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:01.002239 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:01.002303 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.002307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.003338 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.003342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.003989 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.003999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.004482 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.004486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.004555 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.004912 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:01.004926 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:01.005979 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:01.007655 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.007657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.007659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.007884 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.008177 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.008180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.008182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.008280 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.008641 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:01.008644 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:01.008712 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:32:01.008807 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:32:01.008916 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 08:32:01.009095 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.001866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.1.157.124:33557)
0807 08:32:01.001961 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.001965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.001974 (+ 9us) server_negotiation.cc:407] Connection header received
0807 08:32:01.002000 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.002001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.002036 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.002103 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.002421 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.003230 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.004091 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.004385 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.004472 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.006094 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.006098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.006100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.006124 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.007553 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.007994 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.007998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.007999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.008088 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.008368 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.008370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.008371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.008548 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.008634 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.008868 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.008966 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 08:32:01.010442 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:32:01.011515 22135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: Bootstrap starting.
I20260807 08:32:01.013888 22135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:01.014607 22135 log.cc:824] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:01.016311 22135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: No bootstrap required, opened a new log
I20260807 08:32:01.021205 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.012568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.1.157.124:33557)
0807 08:32:01.012673 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.012676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.013112 (+ 436us) server_negotiation.cc:407] Connection header received
0807 08:32:01.013199 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.013202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.013239 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.013312 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.014013 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.014694 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.015691 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.015973 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.016299 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.017455 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.017459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.017461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.017483 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.019017 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.019633 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.019636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.019638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.019749 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.020151 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.020155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.020157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.020375 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.020488 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.020974 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.021063 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:32:01.021340 22135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.021675 22135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:01.021777 22135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974322caf84541489bf03fddd615a269, State: Initialized, Role: FOLLOWER
I20260807 08:32:01.021924 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.012193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.1.157.126:43879)
0807 08:32:01.012285 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.012289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.012503 (+ 214us) server_negotiation.cc:407] Connection header received
0807 08:32:01.012665 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.012668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.012710 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.012771 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.013618 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.014334 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.015398 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.015711 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.015776 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.017486 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.017490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.017492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.017515 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.019528 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.020017 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.020020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.020022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.020102 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.020668 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.020671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.020673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.020865 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.020983 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.021435 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.021792 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:32:01.022269 22135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.022819 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.023018 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.023558 22135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:01.024132 22021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 3
I20260807 08:32:01.025275 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.012879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.1.157.125:37699)
0807 08:32:01.012974 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.012978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.016307 (+ 3329us) server_negotiation.cc:407] Connection header received
0807 08:32:01.016806 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.016809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.016851 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.016898 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.017687 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.018392 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.019409 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.019713 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.019795 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.021216 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.021220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.021223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.021247 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.023456 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.023854 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.023857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.023859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.023934 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.024160 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.024163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.024165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.024366 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.024485 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.025039 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.025163 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 08:32:01.028555 22179 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:32:01.028836 22179 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 08:32:01 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:32:01.026495 22030 master_runner.cc:186] Error getting master registration for 127.1.157.126:43879: OK, instance_id { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" instance_seqno: 1786091520580492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:32:01.029639 22030 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:32:01.032778 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.024528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.1.157.124:33557)
0807 08:32:01.024616 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.024620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.024697 (+ 77us) server_negotiation.cc:407] Connection header received
0807 08:32:01.024797 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.024799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.024838 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.024991 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.025344 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.025987 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.026885 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.027165 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.027559 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.029940 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.029943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.029945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.029964 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.031296 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.031716 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.031719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.031721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.031803 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.032065 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.032068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.032069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.032239 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.032320 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.032568 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.032656 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:32:01.032775 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.024470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36158)
0807 08:32:01.024632 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:01.024644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:01.024730 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:01.024934 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:01.024937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:01.025137 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:01.025226 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.025233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.026110 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.026113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.026746 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.026758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.027290 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.027294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.027371 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.028438 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:01.028452 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:01.029806 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:01.031387 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.031390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.031392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.031599 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.031883 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.031886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.031888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.031970 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.032330 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:01.032333 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:01.032415 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 08:32:01.032497 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:32:01.032618 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 08:32:01.035557 22021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: Bootstrap starting.
I20260807 08:32:01.037755 22021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:01.038422 22021 log.cc:824] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:01.040048 22021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: No bootstrap required, opened a new log
I20260807 08:32:01.040050 21898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } attempt: 3
I20260807 08:32:01.043620 22021 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.043890 22021 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:01.043998 22021 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca953f0209674744b8f5eedff63fc362, State: Initialized, Role: FOLLOWER
I20260807 08:32:01.044471 22021 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.044960 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.045163 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.045619 22021 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 08:32:01 dist-test-slave-6zbq krb5kdc[21772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091520, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:32:01.049531 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.040404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:33557 (local address 127.0.0.1:36164)
0807 08:32:01.040553 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:01.040565 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:01.040653 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:01.040876 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:01.040879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:01.041045 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:01.041150 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.041156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.042040 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.042043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.042899 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:01.042910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.044311 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.044315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:01.044397 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.044823 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:01.044836 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:01.046180 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:01.047948 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.047952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.047954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.048208 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.048549 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:01.048552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:01.048554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:01.048642 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:01.049051 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:01.049054 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:01.049143 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 08:32:01.049251 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 08:32:01.049393 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 08:32:01.050490 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:01.040483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.1.157.124:33557)
0807 08:32:01.040576 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:01.040579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:01.040615 (+ 36us) server_negotiation.cc:407] Connection header received
0807 08:32:01.040746 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:01.040749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:01.040787 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:01.040855 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:01.041263 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.041935 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.043032 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:01.043353 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:01.044276 (+ 923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:01.046321 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:01.046325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:01.046327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:01.046381 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:01.047821 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.048357 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.048361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.048362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.048445 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:01.048742 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:01.048745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:01.048747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:01.048934 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:01.049013 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:01.050230 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0807 08:32:01.050325 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:32:01.049443 21909 master_runner.cc:186] Error getting master registration for 127.1.157.126:43879: OK, instance_id { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" instance_seqno: 1786091520580492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:32:01.052145 21909 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:32:01.052541 22202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:32:01.054418 22202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:32:01.054767 21898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: Bootstrap starting.
I20260807 08:32:01.057147 21898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:01.057930 21898 log.cc:824] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:01.059873 21898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: No bootstrap required, opened a new log
I20260807 08:32:01.064613 21898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.064812 21898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:01.064903 21898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6ba67e6b5046b2b605a73f25f32bf4, State: Initialized, Role: FOLLOWER
I20260807 08:32:01.065330 21898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.065789 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.065968 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.066535 21898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:01.075680 21790 master_runner.cc:263] No leader master found from master 6b6ba67e6b5046b2b605a73f25f32bf4
I20260807 08:32:01.075738 21790 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:32:01.075819 22222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:32:01.075919 22222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 08:32:01.077150 21827 tablet.cc:2369] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:32:01.219331 22064 tablet.cc:2369] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:01.271034 22163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:01.271240 22163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.272096 22163 leader_election.cc:288] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b6ba67e6b5046b2b605a73f25f32bf4 (127.1.157.126:43879), ca953f0209674744b8f5eedff63fc362 (127.1.157.125:37699)
I20260807 08:32:01.272567 21861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "974322caf84541489bf03fddd615a269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" is_pre_election: true
I20260807 08:32:01.272627 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "974322caf84541489bf03fddd615a269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca953f0209674744b8f5eedff63fc362" is_pre_election: true
I20260807 08:32:01.272854 21861 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 974322caf84541489bf03fddd615a269 in term 0.
I20260807 08:32:01.272912 21976 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 974322caf84541489bf03fddd615a269 in term 0.
I20260807 08:32:01.273241 22057 leader_election.cc:302] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b6ba67e6b5046b2b605a73f25f32bf4, 974322caf84541489bf03fddd615a269; no voters:
I20260807 08:32:01.273497 22163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:32:01.273588 22163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 08:32:01.273615 21945 tablet.cc:2369] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:01.273640 22163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:01.274737 22163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.275027 22163 leader_election.cc:288] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [CANDIDATE]: Term 1 election: Requested vote from peers 6b6ba67e6b5046b2b605a73f25f32bf4 (127.1.157.126:43879), ca953f0209674744b8f5eedff63fc362 (127.1.157.125:37699)
I20260807 08:32:01.275308 21861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "974322caf84541489bf03fddd615a269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4"
I20260807 08:32:01.275360 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "974322caf84541489bf03fddd615a269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca953f0209674744b8f5eedff63fc362"
I20260807 08:32:01.275436 21861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:01.275467 21976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:01.276363 21861 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 974322caf84541489bf03fddd615a269 in term 1.
I20260807 08:32:01.276450 21976 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 974322caf84541489bf03fddd615a269 in term 1.
I20260807 08:32:01.276603 22057 leader_election.cc:302] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b6ba67e6b5046b2b605a73f25f32bf4, 974322caf84541489bf03fddd615a269; no voters:
I20260807 08:32:01.276773 22163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:01.277001 22163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [term 1 LEADER]: Becoming Leader. State: Replica: 974322caf84541489bf03fddd615a269, State: Running, Role: LEADER
I20260807 08:32:01.277285 22163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } }
I20260807 08:32:01.279100 22226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974322caf84541489bf03fddd615a269. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.279225 22226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:01.279963 22229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:01.280745 22229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:01.284767 21861 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [term 1 FOLLOWER]: Refusing update from remote peer 974322caf84541489bf03fddd615a269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:01.284767 21976 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [term 1 FOLLOWER]: Refusing update from remote peer 974322caf84541489bf03fddd615a269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:01.285308 22226 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:01.285461 22163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:01.288681 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974322caf84541489bf03fddd615a269. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.288802 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.288739 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974322caf84541489bf03fddd615a269. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.288841 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.291637 22184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.291752 22184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.291770 22229 catalog_manager.cc:1347] Generated new cluster ID: 00e38bfc695a45edaf23ab12b4b681fa
I20260807 08:32:01.291818 22229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:01.292255 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.292356 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:32:01.292320 22233 mvcc.cc:205] Tried to move back new op lower bound from 7315830871176966144 to 7315830871154003968. Current Snapshot: MvccSnapshot[applied={T|T < 7315830871176966144}]
I20260807 08:32:01.292550 22226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.292651 22226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:01.292929 22226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "974322caf84541489bf03fddd615a269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6ba67e6b5046b2b605a73f25f32bf4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43879 } } peers { permanent_uuid: "ca953f0209674744b8f5eedff63fc362" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 37699 } } peers { permanent_uuid: "974322caf84541489bf03fddd615a269" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 33557 } } }
I20260807 08:32:01.293023 22226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:01.304358 22229 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:32:01.305438 22229 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:01.310726 22229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 974322caf84541489bf03fddd615a269: Generated new TSK 0
I20260807 08:32:01.311447 22229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:32:02.055759 22202 catalog_manager.cc:1259] Loaded cluster ID: 00e38bfc695a45edaf23ab12b4b681fa
I20260807 08:32:02.055836 22202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: loading cluster ID for follower catalog manager: success
I20260807 08:32:02.057497 22202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: acquiring CA information for follower catalog manager: success
I20260807 08:32:02.058636 22202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca953f0209674744b8f5eedff63fc362: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:32:02.076920 22222 catalog_manager.cc:1259] Loaded cluster ID: 00e38bfc695a45edaf23ab12b4b681fa
I20260807 08:32:02.076978 22222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: loading cluster ID for follower catalog manager: success
I20260807 08:32:02.078481 22222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: acquiring CA information for follower catalog manager: success
I20260807 08:32:02.079485 22222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b6ba67e6b5046b2b605a73f25f32bf4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:32:02.376372 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 21790
I20260807 08:32:02.390089 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 21909
I20260807 08:32:02.399438 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 22030
2026-08-07T08:32:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3231 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:32:02 dist-test-slave-6zbq krb5kdc[22261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:32:02 dist-test-slave-6zbq krb5kdc[22261](info): set up 2 sockets
Aug 07 08:32:02 dist-test-slave-6zbq krb5kdc[22261](info): commencing operation
krb5kdc: starting...
W20260807 08:32:03.517575 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:32:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:32:03.628734 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42883
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:03.733325 22287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:03.733541 22287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:03.733593 22287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:03.737424 22287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:03.737473 22287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:03.737498 22287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:03.737524 22287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:03.737550 22287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:03.742492 22287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:03.743333 22287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:03.744205 22287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:03.751681 22300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:03.751781 22287 server_base.cc:1034] running on GCE node
W20260807 08:32:03.751686 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:03.751699 22295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:03.752522 22287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:03.753481 22287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:03.754635 22287 hybrid_clock.cc:648] HybridClock initialized: now 1786091523754617 us; error 29 us; skew 500 ppm
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:03.757077 22287 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:03.757846 22287 webserver.cc:460] Webserver started at http://127.1.157.126:41979/ using document root <none> and password file <none>
I20260807 08:32:03.758320 22287 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:03.758419 22287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:03.758611 22287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:03.760452 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "f3c6136698094a5bbb2542289e8f18ee"
format_stamp: "Formatted at 2026-08-07 08:32:03 on dist-test-slave-6zbq"
server_key: "7c2606e6f57041f8169694d0ea0d38bd"
server_key_iv: "c9eedd486a177cab0188721c90656c9f"
server_key_version: "encryptionkey@0"
I20260807 08:32:03.760957 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "f3c6136698094a5bbb2542289e8f18ee"
format_stamp: "Formatted at 2026-08-07 08:32:03 on dist-test-slave-6zbq"
server_key: "7c2606e6f57041f8169694d0ea0d38bd"
server_key_iv: "c9eedd486a177cab0188721c90656c9f"
server_key_version: "encryptionkey@0"
I20260807 08:32:03.764304 22287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 08:32:03.766320 22311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:03.767390 22287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:32:03.767508 22287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "f3c6136698094a5bbb2542289e8f18ee"
format_stamp: "Formatted at 2026-08-07 08:32:03 on dist-test-slave-6zbq"
server_key: "7c2606e6f57041f8169694d0ea0d38bd"
server_key_iv: "c9eedd486a177cab0188721c90656c9f"
server_key_version: "encryptionkey@0"
I20260807 08:32:03.767596 22287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:03.776631 22287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:03.778972 22287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:03.779127 22287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:03.785233 22287 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:42883
I20260807 08:32:03.785279 22394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:42883 every 8 connection(s)
I20260807 08:32:03.786098 22287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:03.788545 22395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:03.793367 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee: Bootstrap starting.
I20260807 08:32:03.793550 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 22287
I20260807 08:32:03.793656 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:32:03.793900 1653 external_mini_cluster.cc:1428] Setting key 560c2cccdf5a6bd23cbcbefac0271297
I20260807 08:32:03.795648 22395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:03.796327 22395 log.cc:824] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:03.798192 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee: No bootstrap required, opened a new log
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:03.802209 22395 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } }
I20260807 08:32:03.802445 22395 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:03.802511 22395 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c6136698094a5bbb2542289e8f18ee, State: Initialized, Role: FOLLOWER
I20260807 08:32:03.802953 22395 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } }
I20260807 08:32:03.803117 22395 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:03.803186 22395 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:03.803289 22395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:03.804090 22395 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } }
I20260807 08:32:03.804399 22395 leader_election.cc:302] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3c6136698094a5bbb2542289e8f18ee; no voters:
I20260807 08:32:03.804680 22395 leader_election.cc:288] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:32:03.804751 22401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:03.804962 22401 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [term 1 LEADER]: Becoming Leader. State: Replica: f3c6136698094a5bbb2542289e8f18ee, State: Running, Role: LEADER
I20260807 08:32:03.805325 22401 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } }
I20260807 08:32:03.805785 22395 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:03.806510 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3c6136698094a5bbb2542289e8f18ee. Latest consensus state: current_term: 1 leader_uuid: "f3c6136698094a5bbb2542289e8f18ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } } }
I20260807 08:32:03.806629 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:03.806983 22411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:03.807492 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:03.794989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.1.157.126:42883)
0807 08:32:03.795342 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:03.795358 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:03.795389 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:32:03.795997 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:03.796014 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:03.796307 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:03.796570 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:03.797363 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:03.798306 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:03.799425 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:03.799870 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:03.799976 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:03.802600 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:03.802605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:03.802616 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:03.802641 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:03.804222 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:03.804634 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:03.804641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:03.804647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:03.804800 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:03.805117 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:03.805120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:03.805122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:03.805606 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:03.805776 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:03.806239 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 08:32:03.806555 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260807 08:32:03.808789 22402 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3c6136698094a5bbb2542289e8f18ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c6136698094a5bbb2542289e8f18ee" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42883 } } }
I20260807 08:32:03.808892 22402 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:03.810108 22411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:03.810420 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:32:03.815609 22411 catalog_manager.cc:1347] Generated new cluster ID: e5fe5c8423a144b4a704b7f8cb286e8e
I20260807 08:32:03.815660 22411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:03.822507 22411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:32:03.823587 22411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:03.839584 22411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3c6136698094a5bbb2542289e8f18ee: Generated new TSK 0
I20260807 08:32:03.840241 22411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:32:03.868058 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 22287
2026-08-07T08:32:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1466 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22441](info): set up 2 sockets
Aug 07 08:32:03 dist-test-slave-6zbq krb5kdc[22441](info): commencing operation
krb5kdc: starting...
W20260807 08:32:04.998706 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:32:05.108696 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:05.213896 22462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:05.214130 22462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:05.214185 22462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:05.218061 22462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:05.218173 22462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:05.218214 22462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:05.218237 22462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:05.218266 22462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:05.223568 22462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:05.224435 22462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:05.225312 22462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:05.233119 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.233125 22462 server_base.cc:1034] running on GCE node
W20260807 08:32:05.233125 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:05.233147 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.233837 22462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:05.234869 22462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:05.236027 22462 hybrid_clock.cc:648] HybridClock initialized: now 1786091525236015 us; error 24 us; skew 500 ppm
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:05.238523 22462 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:05.239326 22462 webserver.cc:460] Webserver started at http://127.1.157.126:36615/ using document root <none> and password file <none>
I20260807 08:32:05.239799 22462 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:05.239866 22462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:05.240041 22462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:05.241858 22462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "c7bcc7dfe907453cb99df7c1d6a95237"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "9b1565d36c5ece677519963f5d816423"
server_key_iv: "be1815eb4b8891f14f63b0d99c9256b9"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.242408 22462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "c7bcc7dfe907453cb99df7c1d6a95237"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "9b1565d36c5ece677519963f5d816423"
server_key_iv: "be1815eb4b8891f14f63b0d99c9256b9"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.245703 22462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:32:05.247843 22483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.248891 22462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:05.249001 22462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c7bcc7dfe907453cb99df7c1d6a95237"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "9b1565d36c5ece677519963f5d816423"
server_key_iv: "be1815eb4b8891f14f63b0d99c9256b9"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.249086 22462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:05.277983 22462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:05.280617 22462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:05.280789 22462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:05.287468 22462 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:35097
I20260807 08:32:05.287725 22560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:35097 every 8 connection(s)
I20260807 08:32:05.288384 22462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:05.290975 22561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:05.293977 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 22462
I20260807 08:32:05.294116 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:32:05.294462 1653 external_mini_cluster.cc:1428] Setting key b13f4ff94674e44d5f33bc1577ab4e09
I20260807 08:32:05.296653 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap starting.
I20260807 08:32:05.298761 22561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:05.299435 22561 log.cc:824] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:05.301393 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: No bootstrap required, opened a new log
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:05.304807 22561 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:05.305014 22561 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:05.305081 22561 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Initialized, Role: FOLLOWER
I20260807 08:32:05.305516 22561 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:05.305665 22561 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:05.305737 22561 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:05.305831 22561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:05.306823 22561 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:05.307147 22561 leader_election.cc:302] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bcc7dfe907453cb99df7c1d6a95237; no voters:
I20260807 08:32:05.307408 22561 leader_election.cc:288] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:32:05.307508 22567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:05.307744 22567 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 LEADER]: Becoming Leader. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Running, Role: LEADER
I20260807 08:32:05.308099 22567 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:05.308636 22561 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:05.309141 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.295824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.1.157.126:35097)
0807 08:32:05.296233 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.296247 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.296281 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:32:05.296931 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.296949 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.297363 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.297658 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:05.298452 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.299463 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.300465 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.300914 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.301034 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.303729 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:05.303734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:05.303746 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:05.303772 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:05.306067 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.306555 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.306560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.306564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.306671 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.306972 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.306975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.306976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.307454 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:05.307657 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.307957 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.308243 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":127,"threads_started":1}
I20260807 08:32:05.309353 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:05.309450 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:05.309643 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcc7dfe907453cb99df7c1d6a95237. Latest consensus state: current_term: 1 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:05.309723 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:05.310060 22576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:05.312927 22576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:05.317939 22576 catalog_manager.cc:1347] Generated new cluster ID: 553b6115430f4e13bee4f8bff8a70132
I20260807 08:32:05.318022 22576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:05.327123 22576 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:32:05.327911 22576 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:05.334744 22576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Generated new TSK 0
I20260807 08:32:05.335310 22576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:32:05.369264 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:05.492465 22597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:05.492683 22597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:05.492722 22597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:05.496621 22597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:05.496673 22597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:05.496726 22597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:32:05.501650 22597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:05.502553 22597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:05.503440 22597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:05.512704 22608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:05.512713 22609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:05.512714 22613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.512759 22597 server_base.cc:1034] running on GCE node
I20260807 08:32:05.513373 22597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:05.513962 22597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:05.515125 22597 hybrid_clock.cc:648] HybridClock initialized: now 1786091525515108 us; error 33 us; skew 500 ppm
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:05.517740 22597 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:32:05.518605 22597 webserver.cc:460] Webserver started at http://127.1.157.65:46815/ using document root <none> and password file <none>
I20260807 08:32:05.519120 22597 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:05.519186 22597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:05.519361 22597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:05.521355 22597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "5d5e7a321b1d448686ffa576c851f3bc"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "f776c383eded33e56ab082624911a19a"
server_key_iv: "e8bf8bc03d308a12c347f685e43141af"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.521875 22597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "5d5e7a321b1d448686ffa576c851f3bc"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "f776c383eded33e56ab082624911a19a"
server_key_iv: "e8bf8bc03d308a12c347f685e43141af"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.525367 22597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:32:05.527596 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.528718 22597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:05.528821 22597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "5d5e7a321b1d448686ffa576c851f3bc"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "f776c383eded33e56ab082624911a19a"
server_key_iv: "e8bf8bc03d308a12c347f685e43141af"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.528909 22597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:05.544162 22597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:05.546999 22597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:05.547163 22597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:05.547688 22597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:05.548448 22597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:05.548496 22597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.548544 22597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:05.548573 22597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.557577 22597 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:46435
I20260807 08:32:05.557605 22800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:46435 every 8 connection(s)
I20260807 08:32:05.558436 22597 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:32:05.565339 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 22597
I20260807 08:32:05.565462 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:32:05.565776 1653 external_mini_cluster.cc:1428] Setting key dd5ce9a9c7c719cf409aa848633b8bb0
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:05.571033 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.560126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:37837 (local address 127.1.157.126:35097)
0807 08:32:05.560285 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.560290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.561265 (+ 975us) server_negotiation.cc:407] Connection header received
0807 08:32:05.562083 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.562085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.562132 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.562204 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:05.563312 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.563844 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.564865 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.565088 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.565182 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.567609 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:05.567613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:05.567615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:05.567649 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:05.568934 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.569545 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.569548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.569549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.569635 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.569965 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.569967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.569968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.570126 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:05.570196 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.570769 (+ 573us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.570888 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 08:32:05.571743 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.560505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:37837)
0807 08:32:05.561130 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.561171 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.561914 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.562377 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.562386 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.562729 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:05.563189 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.563201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.563930 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.563932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.564777 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.564786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.565204 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.565207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.565349 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.565939 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:05.565955 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:05.567447 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:05.569077 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.569083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.569098 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.569436 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.569713 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.569716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.569718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.569889 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.570216 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:05.570222 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:05.570536 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.570699 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.570937 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":113,"threads_started":1}
I20260807 08:32:05.572851 22801 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:05.573071 22801 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:05.573510 22801 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:05.574939 22507 ts_manager.cc:194] Registered new tserver with Master: 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435)
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:32:05.576035 22507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:37837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:32:05.590256 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:05.693820 22806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:05.694026 22806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:05.694065 22806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:05.697929 22806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:05.697973 22806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:05.698036 22806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:32:05.702924 22806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:05.703758 22806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:05.704634 22806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:05.713224 22817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:05.713231 22812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.713382 22806 server_base.cc:1034] running on GCE node
W20260807 08:32:05.713235 22813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.713824 22806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:05.714432 22806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:05.715584 22806 hybrid_clock.cc:648] HybridClock initialized: now 1786091525715571 us; error 27 us; skew 500 ppm
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:05.718068 22806 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:32:05.718878 22806 webserver.cc:460] Webserver started at http://127.1.157.66:39097/ using document root <none> and password file <none>
I20260807 08:32:05.719345 22806 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:05.719400 22806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:05.719602 22806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:05.721597 22806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "41c639be52ec4ade91ea1e6d92e1e534"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "aa262559fbeed421468b9cdedfffa147"
server_key_iv: "37df593af54417f8ae5ed05dd83bc316"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.722108 22806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "41c639be52ec4ade91ea1e6d92e1e534"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "aa262559fbeed421468b9cdedfffa147"
server_key_iv: "37df593af54417f8ae5ed05dd83bc316"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.725477 22806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:32:05.727634 22825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.728690 22806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:05.728786 22806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "41c639be52ec4ade91ea1e6d92e1e534"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "aa262559fbeed421468b9cdedfffa147"
server_key_iv: "37df593af54417f8ae5ed05dd83bc316"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.728880 22806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:05.741169 22806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:05.743912 22806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:05.744061 22806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:05.744555 22806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:05.745265 22806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:05.745311 22806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.745360 22806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:05.745390 22806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.753352 22806 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:33657
I20260807 08:32:05.753387 23003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:33657 every 8 connection(s)
I20260807 08:32:05.754097 22806 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:32:05.754937 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 22806
I20260807 08:32:05.755052 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:32:05.755338 1653 external_mini_cluster.cc:1428] Setting key 800c0f73d1c4fe0b6ca1b6f4f5d58b6d
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:32:05.766738 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.755822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46473 (local address 127.1.157.126:35097)
0807 08:32:05.755943 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.755948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.757163 (+ 1215us) server_negotiation.cc:407] Connection header received
0807 08:32:05.757883 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.757886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.757935 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.758026 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:05.759079 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.759850 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.760882 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.761250 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.761356 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.763704 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:05.763707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:05.763709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:05.763733 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:05.764977 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.765442 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.765444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.765446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.765515 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.765769 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.765773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.765774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.765994 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:05.766075 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.766510 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.766623 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:32:05.767462 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.756194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:46473)
0807 08:32:05.756988 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.757036 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.757701 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.758150 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.758155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.758509 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:05.758941 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.758954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.759955 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.759957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.760765 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.760774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.761412 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.761416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.761551 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.762027 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:05.762042 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:05.763542 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:05.765084 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.765090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.765105 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.765364 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.765577 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.765579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.765580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.765692 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.766074 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:05.766078 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:05.766287 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.766473 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.766735 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":112,"threads_started":1}
I20260807 08:32:05.768436 23005 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:05.768640 23005 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:05.769037 23005 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:05.769932 22507 ts_manager.cc:194] Registered new tserver with Master: 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657)
I20260807 08:32:05.770503 22507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:46473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:32:05.780431 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:05.885149 23012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:05.885363 23012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:05.885401 23012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:05.889312 23012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:05.889358 23012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:05.889427 23012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:32:05.894279 23012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:05.895191 23012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:05.896080 23012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:05.904665 23026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:05.904672 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.904732 23012 server_base.cc:1034] running on GCE node
W20260807 08:32:05.904676 23020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:05.905237 23012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:05.905776 23012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:05.906926 23012 hybrid_clock.cc:648] HybridClock initialized: now 1786091525906910 us; error 25 us; skew 500 ppm
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:05.909466 23012 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:32:05.910274 23012 webserver.cc:460] Webserver started at http://127.1.157.67:38757/ using document root <none> and password file <none>
I20260807 08:32:05.910768 23012 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:05.910827 23012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:05.910995 23012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:05.912832 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "bcfed4c5544f4c7092af69e52930e383"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "b63326f14c45e857d2d2c175e8e86ad0"
server_key_iv: "5bd440542c704db7269ae3867ea14508"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.913337 23012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "bcfed4c5544f4c7092af69e52930e383"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "b63326f14c45e857d2d2c175e8e86ad0"
server_key_iv: "5bd440542c704db7269ae3867ea14508"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.916692 23012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:32:05.918781 23038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.919870 23012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:05.919966 23012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "bcfed4c5544f4c7092af69e52930e383"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "b63326f14c45e857d2d2c175e8e86ad0"
server_key_iv: "5bd440542c704db7269ae3867ea14508"
server_key_version: "encryptionkey@0"
I20260807 08:32:05.920056 23012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:05.938849 23012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:05.941540 23012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:05.941694 23012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:05.942193 23012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:05.942926 23012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:05.942981 23012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.943032 23012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:05.943060 23012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:05.950793 23012 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:35205
I20260807 08:32:05.950845 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:35205 every 8 connection(s)
I20260807 08:32:05.951565 23012 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:32:05.955627 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 23012
I20260807 08:32:05.955727 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:32:05.955971 1653 external_mini_cluster.cc:1428] Setting key 9c190cdb666fc27df8f8eb5fc2c240fa
Aug 07 08:32:05 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:05.963136 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.953056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:49771 (local address 127.1.157.126:35097)
0807 08:32:05.953189 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.953194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.954122 (+ 928us) server_negotiation.cc:407] Connection header received
0807 08:32:05.954932 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.954934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.954980 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.955048 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:05.956124 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.956737 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.958026 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.958248 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.958334 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.960460 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:05.960463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:05.960465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:05.960489 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:05.961560 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.961986 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.961988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.961989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.962051 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.962289 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.962298 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.962299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.962483 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:05.962552 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.962941 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.963025 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:32:05.963816 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.953415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:49771)
0807 08:32:05.953991 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.954029 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.954767 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.955182 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.955188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.955509 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:05.955966 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.955980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.956859 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.956862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.957917 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.957927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.958402 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.958407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.958553 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.958991 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:05.959007 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:05.960313 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:05.961665 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.961669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.961679 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.961914 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.962111 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:05.962113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:05.962114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:05.962228 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:05.962566 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:05.962571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:05.962770 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.962912 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.963128 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":102,"threads_started":1}
I20260807 08:32:05.964752 23214 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:05.964952 23214 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:05.965354 23214 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:05.966235 22507 ts_manager.cc:194] Registered new tserver with Master: bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
I20260807 08:32:05.966768 22507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:49771
I20260807 08:32:05.969269 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:32:05.976836 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.971028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.1.157.126:35097)
0807 08:32:05.971131 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.971135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.971298 (+ 163us) server_negotiation.cc:407] Connection header received
0807 08:32:05.971401 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.971403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.971435 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.971484 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:05.972041 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.972496 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.973196 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.973389 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.973449 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.974539 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:05.974542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:05.974544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:05.974563 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:05.975708 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.976012 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.976014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.976015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.976073 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:05.976243 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:05.976245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:05.976246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:05.976426 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:05.976517 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.976661 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.976735 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:32:05.979094 22507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:32:05.981841 22507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:32:05.994657 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.989750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33657 (local address 127.0.0.1:39504)
0807 08:32:05.990000 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.990014 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.990096 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.990821 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.990823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.990841 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:05.990935 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.990942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.991974 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.991978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.993184 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.993195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994145 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.994148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.994249 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.994285 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.994413 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.994524 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260807 08:32:05.994668 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.989954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.1.157.66:33657)
0807 08:32:05.990371 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.990383 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.990407 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:32:05.990482 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.990487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.990670 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.991061 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:05.991179 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.991858 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.993318 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994031 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.994123 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.994262 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.994442 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.994533 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":94,"threads_started":1}
I20260807 08:32:05.994915 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.989221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35205 (local address 127.0.0.1:33200)
0807 08:32:05.989695 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.989709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.989795 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.990882 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.990886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.990905 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:05.990988 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.990995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.992478 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.992481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.993472 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.993482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994548 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.994552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.994635 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.994661 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.994706 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.994771 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":47,"thread_start_us":89,"threads_started":1}
I20260807 08:32:05.995088 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.989105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46435 (local address 127.0.0.1:49328)
0807 08:32:05.989528 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:05.989571 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:05.989686 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:05.990514 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:05.990520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:05.990544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:05.990670 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.990676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.992417 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.992420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.993640 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:05.993650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994714 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.994717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:05.994794 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.994813 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:32:05.994858 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:32:05.994909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":84,"thread_start_us":64,"threads_started":1}
I20260807 08:32:05.995425 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.990125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.1.157.67:35205)
0807 08:32:05.990478 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.990490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.990509 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:32:05.990567 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.990572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.990744 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.991128 (+ 384us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:05.991257 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.992330 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.993600 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994410 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.995080 (+ 670us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.995193 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.995232 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.995301 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":122,"threads_started":1}
I20260807 08:32:05.995425 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:05.989248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.1.157.65:46435)
0807 08:32:05.989683 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:05.990020 (+ 337us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:05.990051 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:32:05.990131 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:05.990138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:05.990316 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:05.990839 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:05.991070 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.992219 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.993764 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:05.994595 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:05.994989 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:05.995134 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:05.995186 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 08:32:05.995271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":121,"threads_started":1}
I20260807 08:32:05.997841 22908 tablet_service.cc:1467] Processing CreateTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 (DEFAULT_TABLE table=test-table [id=0dc72f56cf56436d98e006793bd3ee26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:05.997838 22706 tablet_service.cc:1467] Processing CreateTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 (DEFAULT_TABLE table=test-table [id=0dc72f56cf56436d98e006793bd3ee26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:05.998768 22706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d97c772c38047cdb0cb8502e2b27a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:05.998781 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d97c772c38047cdb0cb8502e2b27a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:06.000108 23115 tablet_service.cc:1467] Processing CreateTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 (DEFAULT_TABLE table=test-table [id=0dc72f56cf56436d98e006793bd3ee26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:06.001011 23115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d97c772c38047cdb0cb8502e2b27a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:06.004642 23245 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap starting.
I20260807 08:32:06.006685 23251 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Bootstrap starting.
I20260807 08:32:06.007709 23245 tablet_bootstrap.cc:654] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.008116 23252 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Bootstrap starting.
I20260807 08:32:06.008641 23245 log.cc:824] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.009225 23251 tablet_bootstrap.cc:654] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.010013 23251 log.cc:824] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.010135 23252 tablet_bootstrap.cc:654] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.010968 23245 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: No bootstrap required, opened a new log
I20260807 08:32:06.011157 23245 ts_tablet_manager.cc:1397] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 08:32:06.011310 23252 log.cc:824] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.011965 23251 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: No bootstrap required, opened a new log
I20260807 08:32:06.012142 23251 ts_tablet_manager.cc:1397] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260807 08:32:06.013157 23252 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: No bootstrap required, opened a new log
I20260807 08:32:06.013342 23252 ts_tablet_manager.cc:1397] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 08:32:06.014878 23251 raft_consensus.cc:348] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.014884 23245 raft_consensus.cc:348] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.015086 23251 raft_consensus.cc:374] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.015133 23251 raft_consensus.cc:729] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c639be52ec4ade91ea1e6d92e1e534, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.015142 23245 raft_consensus.cc:374] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.015199 23245 raft_consensus.cc:729] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5e7a321b1d448686ffa576c851f3bc, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.015533 23251 consensus_queue.cc:260] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.015615 23245 consensus_queue.cc:260] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.016263 22801 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
I20260807 08:32:06.016397 23251 ts_tablet_manager.cc:1428] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260807 08:32:06.016697 23005 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
I20260807 08:32:06.016865 23252 raft_consensus.cc:348] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.017102 23252 raft_consensus.cc:374] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.017112 23245 ts_tablet_manager.cc:1428] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:32:06.017601 23252 raft_consensus.cc:729] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfed4c5544f4c7092af69e52930e383, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.018006 23252 consensus_queue.cc:260] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.018633 23214 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
I20260807 08:32:06.018882 23252 ts_tablet_manager.cc:1428] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260807 08:32:06.041910 23261 raft_consensus.cc:482] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:06.042038 23261 raft_consensus.cc:504] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.043165 23261 leader_election.cc:288] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205), 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657)
I20260807 08:32:06.046434 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.043362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35205 (local address 127.1.157.65:46655)
0807 08:32:06.043468 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.043482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.043558 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.043887 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.043890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.043907 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:06.044023 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.044029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.044703 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.044706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.045445 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.045454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.046038 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.046041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.046140 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.046159 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.046222 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.046291 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 08:32:06.046434 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.043476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:46655 (local address 127.1.157.67:35205)
0807 08:32:06.043618 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.043622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.043634 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:32:06.043672 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.043675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.043705 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.043767 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:06.044113 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.044604 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.045561 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.045940 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.046038 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.046085 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.046248 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.046320 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:32:06.046811 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.043701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39725 (local address 127.1.157.66:33657)
0807 08:32:06.043805 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.043809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.043957 (+ 148us) server_negotiation.cc:407] Connection header received
0807 08:32:06.044045 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.044047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.044097 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.044168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:06.044430 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.044898 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.045876 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.046268 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.046389 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.046437 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.046645 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.046713 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:32:06.046975 23150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d97c772c38047cdb0cb8502e2b27a07" candidate_uuid: "5d5e7a321b1d448686ffa576c851f3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfed4c5544f4c7092af69e52930e383" is_pre_election: true
I20260807 08:32:06.047148 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.043630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33657 (local address 127.1.157.65:39725)
0807 08:32:06.043874 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.043886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.043975 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.044202 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.044205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.044227 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:06.044335 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.044340 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.044998 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.045001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.045763 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.045772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.046393 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.046397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.046492 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.046517 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.046942 (+ 425us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.047021 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":64,"threads_started":1}
I20260807 08:32:06.047200 23150 raft_consensus.cc:2454] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5e7a321b1d448686ffa576c851f3bc in term 0.
I20260807 08:32:06.047595 22626 leader_election.cc:302] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc, bcfed4c5544f4c7092af69e52930e383; no voters:
I20260807 08:32:06.047653 22934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d97c772c38047cdb0cb8502e2b27a07" candidate_uuid: "5d5e7a321b1d448686ffa576c851f3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c639be52ec4ade91ea1e6d92e1e534" is_pre_election: true
I20260807 08:32:06.047832 23261 raft_consensus.cc:2781] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:32:06.047875 22934 raft_consensus.cc:2454] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d5e7a321b1d448686ffa576c851f3bc in term 0.
I20260807 08:32:06.047911 23261 raft_consensus.cc:482] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:32:06.047961 23261 raft_consensus.cc:3037] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:06.049068 23261 raft_consensus.cc:504] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.049371 23261 leader_election.cc:288] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 1 election: Requested vote from peers bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205), 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657)
I20260807 08:32:06.049615 23150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d97c772c38047cdb0cb8502e2b27a07" candidate_uuid: "5d5e7a321b1d448686ffa576c851f3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfed4c5544f4c7092af69e52930e383"
I20260807 08:32:06.049726 22934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d97c772c38047cdb0cb8502e2b27a07" candidate_uuid: "5d5e7a321b1d448686ffa576c851f3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c639be52ec4ade91ea1e6d92e1e534"
I20260807 08:32:06.049723 23150 raft_consensus.cc:3037] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:06.049829 22934 raft_consensus.cc:3037] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:06.050891 22934 raft_consensus.cc:2454] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5e7a321b1d448686ffa576c851f3bc in term 1.
I20260807 08:32:06.050891 23150 raft_consensus.cc:2454] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d5e7a321b1d448686ffa576c851f3bc in term 1.
I20260807 08:32:06.051138 22626 leader_election.cc:302] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c639be52ec4ade91ea1e6d92e1e534, 5d5e7a321b1d448686ffa576c851f3bc; no voters:
I20260807 08:32:06.051283 23261 raft_consensus.cc:2781] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:06.051476 23261 raft_consensus.cc:686] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 LEADER]: Becoming Leader. State: Replica: 5d5e7a321b1d448686ffa576c851f3bc, State: Running, Role: LEADER
I20260807 08:32:06.051750 23261 consensus_queue.cc:237] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.054414 22507 catalog_manager.cc:5495] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65). New cstate: current_term: 1 leader_uuid: "5d5e7a321b1d448686ffa576c851f3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } health_report { overall_health: UNKNOWN } } }
W20260807 08:32:06.059433 22802 tablet.cc:2369] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:06.071188 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.068059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.1.157.65:46435)
0807 08:32:06.068160 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.068164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.068175 (+ 11us) server_negotiation.cc:407] Connection header received
0807 08:32:06.068309 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.068311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.068355 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.068430 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:06.068589 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.069026 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.069894 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.070096 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.070213 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.070498 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:06.070858 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:32:06.070947 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.071001 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.071042 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:32:06.075943 23150 raft_consensus.cc:1261] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Refusing update from remote peer 5d5e7a321b1d448686ffa576c851f3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:06.075943 22934 raft_consensus.cc:1261] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Refusing update from remote peer 5d5e7a321b1d448686ffa576c851f3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:06.076572 23261 consensus_queue.cc:1035] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:06.076758 23264 consensus_queue.cc:1035] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:06.083300 23267 mvcc.cc:205] Tried to move back new op lower bound from 7315830890801414144 to 7315830890710269952. Current Snapshot: MvccSnapshot[applied={T|T < 7315830890801414144}]
I20260807 08:32:06.084623 23270 mvcc.cc:205] Tried to move back new op lower bound from 7315830890801414144 to 7315830890710269952. Current Snapshot: MvccSnapshot[applied={T|T < 7315830890801414144}]
I20260807 08:32:06.084800 23268 mvcc.cc:205] Tried to move back new op lower bound from 7315830890801414144 to 7315830890710269952. Current Snapshot: MvccSnapshot[applied={T|T < 7315830890801414144}]
I20260807 08:32:06.092183 22504 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:06.092303 22504 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:06.094537 22504 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Sending DeleteTablet for 3 replicas of tablet 1d97c772c38047cdb0cb8502e2b27a07
I20260807 08:32:06.095204 22706 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:06 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:49328
I20260807 08:32:06.095350 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:06 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:39504
I20260807 08:32:06.095741 23284 tablet_replica.cc:330] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: stopping tablet replica
I20260807 08:32:06.095745 23115 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:06 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33200
I20260807 08:32:06.096031 23284 raft_consensus.cc:2229] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:06.096228 23285 tablet_replica.cc:330] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: stopping tablet replica
I20260807 08:32:06.096309 23284 raft_consensus.cc:2258] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:06.096374 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 22597
I20260807 08:32:06.098737 23285 raft_consensus.cc:2229] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:06.098982 23285 raft_consensus.cc:2258] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:06.103668 23285 ts_tablet_manager.cc:1905] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:06.105859 23285 ts_tablet_manager.cc:1918] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:06.106002 23285 log.cc:1196] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/1d97c772c38047cdb0cb8502e2b27a07
I20260807 08:32:06.106401 23285 ts_tablet_manager.cc:1939] T 1d97c772c38047cdb0cb8502e2b27a07 P bcfed4c5544f4c7092af69e52930e383: Deleting consensus metadata
I20260807 08:32:06.106874 23284 ts_tablet_manager.cc:1905] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:06.108193 22487 catalog_manager.cc:4841] TS bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): tablet 1d97c772c38047cdb0cb8502e2b27a07 (table test-table [id=0dc72f56cf56436d98e006793bd3ee26]) successfully deleted
I20260807 08:32:06.109061 23284 ts_tablet_manager.cc:1918] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:06.109202 23284 log.cc:1196] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/1d97c772c38047cdb0cb8502e2b27a07
I20260807 08:32:06.109459 23284 ts_tablet_manager.cc:1939] T 1d97c772c38047cdb0cb8502e2b27a07 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting consensus metadata
I20260807 08:32:06.110425 22487 catalog_manager.cc:4841] TS 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): tablet 1d97c772c38047cdb0cb8502e2b27a07 (table test-table [id=0dc72f56cf56436d98e006793bd3ee26]) successfully deleted
W20260807 08:32:06.114418 22491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:46435 (error 108)
I20260807 08:32:06.114439 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 22806
I20260807 08:32:06.116833 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.116107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46435 (local address 127.0.0.1:49350)
0807 08:32:06.116676 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.116753 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
W20260807 08:32:06.117924 22491 catalog_manager.cc:4584] TS 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d97c772c38047cdb0cb8502e2b27a07: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46435: connect: Connection refused (error 111)
I20260807 08:32:06.124425 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 23012
I20260807 08:32:06.136993 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 22462
I20260807 08:32:06.146922 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--webserver_interface=127.1.157.126
--webserver_port=36615
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:06.251087 23288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:06.251295 23288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:06.251332 23288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:06.255172 23288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:06.255216 23288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:06.255234 23288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:06.255249 23288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:06.255266 23288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:06.260171 23288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=36615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:06.261039 23288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:06.261965 23288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:06.275511 23297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.275671 23288 server_base.cc:1034] running on GCE node
W20260807 08:32:06.275581 23298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:06.275511 23301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.276276 23288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:06.277289 23288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:06.278391 23288 hybrid_clock.cc:648] HybridClock initialized: now 1786091526278366 us; error 39 us; skew 500 ppm
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:06.280965 23288 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:06.281805 23288 webserver.cc:460] Webserver started at http://127.1.157.126:36615/ using document root <none> and password file <none>
I20260807 08:32:06.282313 23288 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:06.282428 23288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:06.285969 23288 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:06.287910 23312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.289072 23288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:06.289191 23288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c7bcc7dfe907453cb99df7c1d6a95237"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "9b1565d36c5ece677519963f5d816423"
server_key_iv: "be1815eb4b8891f14f63b0d99c9256b9"
server_key_version: "encryptionkey@0"
I20260807 08:32:06.289747 23288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:06.300678 23288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:06.303655 23288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:06.303835 23288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:06.310640 23288 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:35097
I20260807 08:32:06.310882 23382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:35097 every 8 connection(s)
I20260807 08:32:06.311698 23288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:06.312539 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 23288
I20260807 08:32:06.313030 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:46435
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=46815
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:06.316668 23383 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:32:06.318307 23383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap starting.
I20260807 08:32:06.335256 23383 log.cc:824] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.343355 23383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:06.343648 23383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap complete.
I20260807 08:32:06.347440 23383 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:06.347831 23383 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.348272 23383 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:06.348378 23383 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:06.348441 23383 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:06.348526 23383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:32:06.350253 23383 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:06.350557 23383 leader_election.cc:302] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bcc7dfe907453cb99df7c1d6a95237; no voters:
I20260807 08:32:06.350831 23383 leader_election.cc:288] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:32:06.350906 23396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:32:06.351085 23396 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 LEADER]: Becoming Leader. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Running, Role: LEADER
I20260807 08:32:06.351357 23396 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:06.351987 23383 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:06.352270 23396 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcc7dfe907453cb99df7c1d6a95237. Latest consensus state: current_term: 2 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:06.352466 23396 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:06.353067 23397 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:06.353232 23397 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:06.353750 23403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:06.358083 23403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:06.358716 23403 catalog_manager.cc:1259] Loaded cluster ID: 553b6115430f4e13bee4f8bff8a70132
I20260807 08:32:06.358775 23403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:06.360119 23403 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 08:32:06.365132 23403 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 08:32:06.369983 23403 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:06.370672 23403 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Loaded TSK: 0
I20260807 08:32:06.371762 23403 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:32:06.423059 23391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:06.423295 23391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:06.423349 23391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:06.427239 23391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:06.427290 23391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:06.427362 23391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:32:06.432341 23391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:46435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=46815
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:06.433221 23391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:06.434131 23391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:06.442986 23433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:06.442984 23434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.443228 23391 server_base.cc:1034] running on GCE node
W20260807 08:32:06.443349 23438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.443735 23391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:06.444321 23391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:06.445495 23391 hybrid_clock.cc:648] HybridClock initialized: now 1786091526445485 us; error 31 us; skew 500 ppm
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:06.447988 23391 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:32:06.448822 23391 webserver.cc:460] Webserver started at http://127.1.157.65:46815/ using document root <none> and password file <none>
I20260807 08:32:06.449316 23391 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:06.449388 23391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:06.453016 23391 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:06.455139 23447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.456305 23391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 08:32:06.456394 23391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "5d5e7a321b1d448686ffa576c851f3bc"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "f776c383eded33e56ab082624911a19a"
server_key_iv: "e8bf8bc03d308a12c347f685e43141af"
server_key_version: "encryptionkey@0"
I20260807 08:32:06.456905 23391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:06.474067 23391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:06.476967 23391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:06.477137 23391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:06.477695 23391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:06.478837 23455 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:32:06.483955 23391 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:32:06.484011 23391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 08:32:06.484097 23391 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:32:06.485626 23391 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:32:06.485671 23391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 08:32:06.485958 23455 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap starting.
I20260807 08:32:06.495198 23391 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:46435
I20260807 08:32:06.496027 23391 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:32:06.496757 23626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:46435 every 8 connection(s)
I20260807 08:32:06.499884 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 23391
I20260807 08:32:06.500335 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:33657
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=39097
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:06.507418 23455 log.cc:824] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:06.514752 23455 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:06.515131 23455 tablet_bootstrap.cc:492] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap complete.
I20260807 08:32:06.515302 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.497949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:58899)
0807 08:32:06.498779 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.498834 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.499903 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.501513 (+ 1610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.501521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.501966 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:06.502623 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.502646 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.504359 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.504363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.505764 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.505777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.506799 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.506803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.507015 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.507681 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:06.507703 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:06.509556 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:06.512024 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.512030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.512043 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.512381 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.512783 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.512787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.512789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.512951 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.513605 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:06.513610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:06.513869 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.514057 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.514396 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":165,"threads_started":1}
I20260807 08:32:06.516040 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.498074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:58899 (local address 127.1.157.126:35097)
0807 08:32:06.498468 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.498487 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.498953 (+ 466us) server_negotiation.cc:407] Connection header received
0807 08:32:06.500340 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.500355 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.500733 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.501062 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:06.502814 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.504188 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.505909 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.506619 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.506775 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.509771 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:06.509778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:06.509795 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:06.509829 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:06.511893 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.512504 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.512510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.512515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.512675 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.513050 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.513054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.513055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.513483 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:06.513613 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.514188 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.514497 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":115,"threads_started":1}
I20260807 08:32:06.516392 23455 ts_tablet_manager.cc:1397] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent bootstrapping tablet: real 0.031s user 0.023s sys 0.004s
I20260807 08:32:06.518303 23628 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:06.518555 23628 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:06.519055 23628 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:06.519049 23455 raft_consensus.cc:348] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.519490 23455 raft_consensus.cc:729] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5e7a321b1d448686ffa576c851f3bc, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.519907 23455 consensus_queue.cc:260] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } }
I20260807 08:32:06.520653 23455 ts_tablet_manager.cc:1428] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260807 08:32:06.522257 23332 ts_manager.cc:194] Registered new tserver with Master: 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435)
I20260807 08:32:06.525321 23332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:58899
I20260807 08:32:06.528265 23628 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.65@KRBTEST.COM
I20260807 08:32:06.533994 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.524730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46435 (local address 127.0.0.1:49358)
0807 08:32:06.525391 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.525436 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.525574 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.526188 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.526193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.526594 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:06.526700 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.526706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.527617 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.527628 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.528216 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.528225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.528854 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.528858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.528963 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.529416 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:06.529427 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:06.530788 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:06.532441 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.532450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.532456 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.532669 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.532940 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.532943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.532945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.533021 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.533432 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:06.533437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:06.533543 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.533661 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.533833 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":98,"threads_started":1}
I20260807 08:32:06.534032 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.524814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.1.157.65:46435)
0807 08:32:06.525144 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.525156 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.525532 (+ 376us) server_negotiation.cc:407] Connection header received
0807 08:32:06.525704 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.525710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.525894 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.526027 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:06.526811 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.527513 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.528323 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.528750 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.528839 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.530912 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:06.530921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:06.530933 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:06.530955 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:06.532341 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.532772 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.532777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.532779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.532857 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.533098 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.533101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.533103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.533323 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:06.533436 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.533755 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.533895 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
I20260807 08:32:06.535141 23517 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d97c772c38047cdb0cb8502e2b27a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:06 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:49358
I20260807 08:32:06.535909 23641 tablet_replica.cc:330] stopping tablet replica
I20260807 08:32:06.536201 23641 raft_consensus.cc:2229] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:06.536428 23641 raft_consensus.cc:2258] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:06.537434 23641 ts_tablet_manager.cc:1905] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:06.541630 23641 ts_tablet_manager.cc:1918] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:06.541724 23641 log.cc:1196] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/1d97c772c38047cdb0cb8502e2b27a07
I20260807 08:32:06.541996 23641 ts_tablet_manager.cc:1939] T 1d97c772c38047cdb0cb8502e2b27a07 P 5d5e7a321b1d448686ffa576c851f3bc: Deleting consensus metadata
I20260807 08:32:06.543720 23316 catalog_manager.cc:4841] TS 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435): tablet 1d97c772c38047cdb0cb8502e2b27a07 (table test-table [id=0dc72f56cf56436d98e006793bd3ee26]) successfully deleted
W20260807 08:32:06.619310 23636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:06.619535 23636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:06.619591 23636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:06.623512 23636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:06.623561 23636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:06.623636 23636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:32:06.628647 23636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:33657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=39097
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:06.629575 23636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:06.630553 23636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:32:06.639488 23636 server_base.cc:1034] running on GCE node
W20260807 08:32:06.639457 23653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:06.639685 23652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:06.639420 23656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.640146 23636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:06.640713 23636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:06.641897 23636 hybrid_clock.cc:648] HybridClock initialized: now 1786091526641871 us; error 33 us; skew 500 ppm
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:06.644424 23636 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:32:06.645200 23636 webserver.cc:460] Webserver started at http://127.1.157.66:39097/ using document root <none> and password file <none>
I20260807 08:32:06.645710 23636 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:06.645788 23636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:06.649497 23636 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:06.651369 23665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.652505 23636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:06.652607 23636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "41c639be52ec4ade91ea1e6d92e1e534"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "aa262559fbeed421468b9cdedfffa147"
server_key_iv: "37df593af54417f8ae5ed05dd83bc316"
server_key_version: "encryptionkey@0"
I20260807 08:32:06.653158 23636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:06.677103 23636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:06.679641 23636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:06.679798 23636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:06.680313 23636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:06.681046 23636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:06.681105 23636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.681155 23636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:06.681185 23636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.690968 23636 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:33657
I20260807 08:32:06.690999 23848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:33657 every 8 connection(s)
I20260807 08:32:06.691825 23636 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:32:06.698479 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 23636
I20260807 08:32:06.698822 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:35205
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=38757
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:06.704603 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.693464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:51665 (local address 127.1.157.126:35097)
0807 08:32:06.693571 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.693574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.694382 (+ 808us) server_negotiation.cc:407] Connection header received
0807 08:32:06.695616 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.695619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.695680 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.695750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:06.696811 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.697329 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.698428 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.698690 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.698783 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.701151 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:06.701154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:06.701157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:06.701185 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:06.702619 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.703171 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.703174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.703175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.703250 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.703576 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.703578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.703579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.703730 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:06.703801 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.704384 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.704486 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:32:06.705636 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.693828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:51665)
0807 08:32:06.694231 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.694270 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.695427 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.695929 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.695936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.696251 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:06.696673 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.696686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.697446 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.697449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.698300 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.698310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.698832 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.698837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.699017 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.699516 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:06.699531 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:06.700992 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:06.702746 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.702752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.702765 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.703079 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.703331 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.703333 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.703335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.703501 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.703835 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:06.703842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:06.704133 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.704337 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.704668 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260807 08:32:06.706925 23850 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:06.707125 23850 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:06.707540 23850 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:06.708547 23332 ts_manager.cc:194] Registered new tserver with Master: 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657)
I20260807 08:32:06.709280 23332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:51665
W20260807 08:32:06.807644 23855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:06.807852 23855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:06.807889 23855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:06.811687 23855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:06.811731 23855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:06.811784 23855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:32:06.816603 23855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:35205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=38757
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:06.817390 23855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:06.818207 23855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:06.826589 23864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:06.826637 23870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.826680 23855 server_base.cc:1034] running on GCE node
W20260807 08:32:06.826649 23865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:06.827193 23855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:06.827706 23855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:06.828855 23855 hybrid_clock.cc:648] HybridClock initialized: now 1786091526828838 us; error 26 us; skew 500 ppm
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:06.831360 23855 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:32:06.832068 23855 webserver.cc:460] Webserver started at http://127.1.157.67:38757/ using document root <none> and password file <none>
I20260807 08:32:06.832510 23855 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:06.832576 23855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:06.835981 23855 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:06.837764 23882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.838852 23855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:06.838939 23855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "bcfed4c5544f4c7092af69e52930e383"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "b63326f14c45e857d2d2c175e8e86ad0"
server_key_iv: "5bd440542c704db7269ae3867ea14508"
server_key_version: "encryptionkey@0"
I20260807 08:32:06.839371 23855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:06.854908 23855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:06.857532 23855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:06.857661 23855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:06.858100 23855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:06.858816 23855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:06.858860 23855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.858894 23855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:06.858913 23855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:06.866730 23855 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:35205
I20260807 08:32:06.866783 24047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:35205 every 8 connection(s)
I20260807 08:32:06.867579 23855 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:32:06 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:06.876271 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 23855
I20260807 08:32:06.879218 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.869085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:34313 (local address 127.1.157.126:35097)
0807 08:32:06.869212 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.869215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.870157 (+ 942us) server_negotiation.cc:407] Connection header received
0807 08:32:06.870943 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.870946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.870991 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.871059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:06.872021 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.872632 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.873638 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.873855 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.873931 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.875958 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:06.875961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:06.875963 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:06.875987 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:06.877413 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.877855 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.877859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.877861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.877934 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.878230 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.878234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.878235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.878450 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:06.878532 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.878959 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.879067 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:32:06.879864 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.869456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:34313)
0807 08:32:06.870032 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.870067 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.870770 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.871187 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.871193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.871496 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:06.871911 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.871924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.872726 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.872728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.873531 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.873539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.873962 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.873964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.874096 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.874536 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:06.874551 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:06.875817 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:06.877516 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.877522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.877534 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.877758 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.878014 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:06.878017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:06.878019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:06.878144 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:06.878544 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:06.878550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:06.878745 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.878899 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.879168 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":107,"threads_started":1}
I20260807 08:32:06.880957 24048 heartbeater.cc:347] Connected to a master server at 127.1.157.126:35097
I20260807 08:32:06.881157 24048 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:06.881635 24048 heartbeater.cc:511] Master 127.1.157.126:35097 requested a full tablet report, sending...
I20260807 08:32:06.882611 23332 ts_manager.cc:194] Registered new tserver with Master: bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
I20260807 08:32:06.883260 23332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:34313
I20260807 08:32:06.884280 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.876854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.1.157.126:35097)
0807 08:32:06.877071 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.877075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.877224 (+ 149us) server_negotiation.cc:407] Connection header received
0807 08:32:06.877338 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.877341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.877382 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.877453 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:06.878125 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.878678 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.879589 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.879810 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.879891 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.881316 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:06.881320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:06.881323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:06.881347 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:06.882782 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.883203 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.883205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.883206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.883269 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:06.883541 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:06.883544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:06.883546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:06.883710 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:06.883897 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.884020 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.884127 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260807 08:32:06.885767 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:32:06.889480 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.886272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.1.157.126:35097)
0807 08:32:06.886404 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.886407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.886553 (+ 146us) server_negotiation.cc:407] Connection header received
0807 08:32:06.886679 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.886682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.886756 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.886834 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:06.887040 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.887499 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.888311 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.888520 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.888603 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.888878 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:06.889165 (+ 287us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:32:06.889260 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.889331 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.889380 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:32:06.889997 23332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:32:06.893013 23332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:32:06.899634 23517 tablet_service.cc:1467] Processing CreateTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 (DEFAULT_TABLE table=test-table [id=6c3208b391b043c2a8345cb40b90aa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:06.900247 23517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c80440f11694dc9a1511d9b03e8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:06.902977 23455 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap starting.
I20260807 08:32:06.903728 23455 tablet_bootstrap.cc:654] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.904592 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.898640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33657 (local address 127.0.0.1:39520)
0807 08:32:06.898740 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.898754 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.898820 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.900037 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.900040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.900059 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:06.900168 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.900175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.901869 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.901872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.902729 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.902739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.904096 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.904100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.904195 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.904303 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.904370 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.904445 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":41}
I20260807 08:32:06.904644 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.899202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35205 (local address 127.0.0.1:33210)
0807 08:32:06.899463 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:06.899477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:06.899565 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:06.900214 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:06.900217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:06.900234 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:06.900325 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.900332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.901624 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.901628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.902814 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:06.902825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.904221 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.904225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:06.904324 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.904344 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:32:06.904412 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:32:06.904483 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":29,"thread_start_us":108,"threads_started":1}
I20260807 08:32:06.904670 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.898974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.1.157.66:33657)
0807 08:32:06.899580 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.899595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.899617 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:32:06.899691 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.899696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.899888 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.900041 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:06.900303 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.901753 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.902840 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.903632 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.904281 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.904413 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.904463 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.904544 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":154,"threads_started":1}
I20260807 08:32:06.905014 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:06.899349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.1.157.67:35205)
0807 08:32:06.899750 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:06.899758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:06.899774 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:32:06.899834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:06.899839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:06.900004 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:06.900147 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:06.900421 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.901497 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.903152 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:06.903927 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:06.904029 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:06.904164 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:06.904798 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 08:32:06.904889 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":143,"threads_started":1}
I20260807 08:32:06.906083 23455 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: No bootstrap required, opened a new log
I20260807 08:32:06.906169 23455 ts_tablet_manager.cc:1397] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 08:32:06.906591 23455 raft_consensus.cc:348] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.906687 23455 raft_consensus.cc:374] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.906723 23455 raft_consensus.cc:729] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5e7a321b1d448686ffa576c851f3bc, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.906845 23455 consensus_queue.cc:260] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.907083 23455 ts_tablet_manager.cc:1428] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:32:06.907480 23951 tablet_service.cc:1467] Processing CreateTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 (DEFAULT_TABLE table=test-table [id=6c3208b391b043c2a8345cb40b90aa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:06.907480 23749 tablet_service.cc:1467] Processing CreateTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 (DEFAULT_TABLE table=test-table [id=6c3208b391b043c2a8345cb40b90aa04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:06.908497 23749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c80440f11694dc9a1511d9b03e8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:06.908504 23951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c80440f11694dc9a1511d9b03e8c3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:06.916082 24065 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Bootstrap starting.
I20260807 08:32:06.916507 24066 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Bootstrap starting.
I20260807 08:32:06.918637 24066 tablet_bootstrap.cc:654] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.919309 24066 log.cc:824] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.920356 24065 tablet_bootstrap.cc:654] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:06.920976 24065 log.cc:824] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:06.921006 24066 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: No bootstrap required, opened a new log
I20260807 08:32:06.921182 24066 ts_tablet_manager.cc:1397] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:32:06.922511 24065 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: No bootstrap required, opened a new log
I20260807 08:32:06.922673 24065 ts_tablet_manager.cc:1397] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 08:32:06.923913 24066 raft_consensus.cc:348] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.924119 24066 raft_consensus.cc:374] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.924162 24066 raft_consensus.cc:729] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfed4c5544f4c7092af69e52930e383, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.924538 24066 consensus_queue.cc:260] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.925097 24048 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
I20260807 08:32:06.925609 24066 ts_tablet_manager.cc:1428] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 08:32:06.926362 24065 raft_consensus.cc:348] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.926635 24065 raft_consensus.cc:374] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:06.926702 24065 raft_consensus.cc:729] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c639be52ec4ade91ea1e6d92e1e534, State: Initialized, Role: FOLLOWER
I20260807 08:32:06.927089 24065 consensus_queue.cc:260] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:06.927673 23850 heartbeater.cc:503] Master 127.1.157.126:35097 was elected leader, sending a full tablet report...
I20260807 08:32:06.928000 24065 ts_tablet_manager.cc:1428] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260807 08:32:06.942615 23852 tablet.cc:2369] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 08:32:06.997160 23630 tablet.cc:2369] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:07.012599 24075 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:07.012810 24075 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:07.013902 24075 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205), 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435)
I20260807 08:32:07.017845 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.014147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35205 (local address 127.1.157.66:46945)
0807 08:32:07.014273 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.014288 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:07.014397 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:07.014687 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:07.014691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:07.014717 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:07.014841 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:07.014849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.015789 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.015791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:07.016825 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:07.016836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.017514 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.017517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:07.017597 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:07.017617 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:32:07.017683 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:32:07.017740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 08:32:07.017899 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.014233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:46945 (local address 127.1.157.67:35205)
0807 08:32:07.014390 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:07.014394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:07.014410 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:32:07.014501 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:07.014503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:07.014553 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:07.014626 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:07.014961 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.015677 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.016958 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.017408 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.017521 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:07.017574 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:07.017708 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 08:32:07.017774 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:32:07.018481 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.014160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46435 (local address 127.1.157.66:38419)
0807 08:32:07.014785 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.014798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:07.014893 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:07.015160 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:07.015162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:07.015176 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:07.015267 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:07.015271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.016224 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.016228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:07.017208 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:07.017217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.018120 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.018123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:07.018217 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:07.018238 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:32:07.018295 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 08:32:07.018370 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":318,"thread_start_us":119,"threads_started":1}
I20260807 08:32:07.018534 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.014304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:38419 (local address 127.1.157.65:46435)
0807 08:32:07.014467 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:07.014470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:07.014888 (+ 418us) server_negotiation.cc:407] Connection header received
0807 08:32:07.014973 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:07.014976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:07.015040 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:07.015141 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:07.015362 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.016102 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.017324 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.017922 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.018033 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:07.018184 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:07.018327 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 08:32:07.018424 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 08:32:07.018515 23975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c80440f11694dc9a1511d9b03e8c3e1" candidate_uuid: "41c639be52ec4ade91ea1e6d92e1e534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfed4c5544f4c7092af69e52930e383" is_pre_election: true
I20260807 08:32:07.018843 23975 raft_consensus.cc:2454] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c639be52ec4ade91ea1e6d92e1e534 in term 0.
I20260807 08:32:07.019059 23553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c80440f11694dc9a1511d9b03e8c3e1" candidate_uuid: "41c639be52ec4ade91ea1e6d92e1e534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5e7a321b1d448686ffa576c851f3bc" is_pre_election: true
I20260807 08:32:07.019317 23671 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383; no voters:
I20260807 08:32:07.019357 23553 raft_consensus.cc:2454] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c639be52ec4ade91ea1e6d92e1e534 in term 0.
I20260807 08:32:07.019598 24075 raft_consensus.cc:2781] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:32:07.019712 24075 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:32:07.019770 24075 raft_consensus.cc:3037] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:07.020756 24075 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:07.021061 24075 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [CANDIDATE]: Term 1 election: Requested vote from peers bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205), 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435)
I20260807 08:32:07.021430 23975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c80440f11694dc9a1511d9b03e8c3e1" candidate_uuid: "41c639be52ec4ade91ea1e6d92e1e534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcfed4c5544f4c7092af69e52930e383"
I20260807 08:32:07.021425 23553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c80440f11694dc9a1511d9b03e8c3e1" candidate_uuid: "41c639be52ec4ade91ea1e6d92e1e534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5e7a321b1d448686ffa576c851f3bc"
I20260807 08:32:07.021543 23975 raft_consensus.cc:3037] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:07.021544 23553 raft_consensus.cc:3037] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:07.022678 23553 raft_consensus.cc:2454] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c639be52ec4ade91ea1e6d92e1e534 in term 1.
I20260807 08:32:07.022686 23975 raft_consensus.cc:2454] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c639be52ec4ade91ea1e6d92e1e534 in term 1.
I20260807 08:32:07.022936 23675 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c639be52ec4ade91ea1e6d92e1e534, 5d5e7a321b1d448686ffa576c851f3bc; no voters:
I20260807 08:32:07.023124 24075 raft_consensus.cc:2781] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:07.023344 24075 raft_consensus.cc:686] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 LEADER]: Becoming Leader. State: Replica: 41c639be52ec4ade91ea1e6d92e1e534, State: Running, Role: LEADER
I20260807 08:32:07.023700 24075 consensus_queue.cc:237] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:07.026678 23332 catalog_manager.cc:5495] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "41c639be52ec4ade91ea1e6d92e1e534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } health_report { overall_health: UNKNOWN } } }
I20260807 08:32:07.058929 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.055506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.1.157.66:33657)
0807 08:32:07.055628 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:07.055632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:07.055645 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:32:07.055742 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:07.055745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:07.055794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:07.055880 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:07.056140 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.056663 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.057569 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:07.057784 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:07.057868 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:07.058158 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:07.058575 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:32:07.058678 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:07.058751 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:32:07.058805 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 08:32:07.064344 23553 raft_consensus.cc:1261] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Refusing update from remote peer 41c639be52ec4ade91ea1e6d92e1e534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:07.064350 23975 raft_consensus.cc:1261] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Refusing update from remote peer 41c639be52ec4ade91ea1e6d92e1e534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:07.064967 24077 consensus_queue.cc:1035] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:07.065132 24075 consensus_queue.cc:1035] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:07.070317 24082 mvcc.cc:205] Tried to move back new op lower bound from 7315830894848856064 to 7315830894691332096. Current Snapshot: MvccSnapshot[applied={T|T < 7315830894848856064}]
I20260807 08:32:07.072739 24085 mvcc.cc:205] Tried to move back new op lower bound from 7315830894848856064 to 7315830894691332096. Current Snapshot: MvccSnapshot[applied={T|T < 7315830894848856064}]
I20260807 08:32:07.078774 23331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:07.078917 23331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:07.080947 23331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Sending DeleteTablet for 3 replicas of tablet 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:07.081525 23517 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:07 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:49358
I20260807 08:32:07.081621 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 23391
I20260807 08:32:07.081599 23951 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:07 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:33210
I20260807 08:32:07.095992 24094 tablet_replica.cc:330] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: stopping tablet replica
I20260807 08:32:07.096010 23749 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c80440f11694dc9a1511d9b03e8c3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:07 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:39520
I20260807 08:32:07.096352 24094 raft_consensus.cc:2229] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:07.096643 24094 raft_consensus.cc:2258] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:07.100075 24095 tablet_replica.cc:330] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: stopping tablet replica
I20260807 08:32:07.100400 24095 raft_consensus.cc:2229] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:32:07.100839 24095 raft_consensus.cc:2258] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:07.101215 24094 ts_tablet_manager.cc:1905] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:07.102237 24095 ts_tablet_manager.cc:1905] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:07.104148 24094 ts_tablet_manager.cc:1918] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:07.104321 24094 log.cc:1196] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:07.104718 24094 ts_tablet_manager.cc:1939] T 7c80440f11694dc9a1511d9b03e8c3e1 P bcfed4c5544f4c7092af69e52930e383: Deleting consensus metadata
I20260807 08:32:07.104815 24095 ts_tablet_manager.cc:1918] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:07.104962 24095 log.cc:1196] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:07.105327 24095 ts_tablet_manager.cc:1939] T 7c80440f11694dc9a1511d9b03e8c3e1 P 41c639be52ec4ade91ea1e6d92e1e534: Deleting consensus metadata
I20260807 08:32:07.106031 23314 catalog_manager.cc:4841] TS bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): tablet 7c80440f11694dc9a1511d9b03e8c3e1 (table test-table [id=6c3208b391b043c2a8345cb40b90aa04]) successfully deleted
I20260807 08:32:07.106356 23314 catalog_manager.cc:4841] TS 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): tablet 7c80440f11694dc9a1511d9b03e8c3e1 (table test-table [id=6c3208b391b043c2a8345cb40b90aa04]) successfully deleted
W20260807 08:32:07.106892 23316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:46435 (error 108)
I20260807 08:32:07.107172 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 23636
I20260807 08:32:07.117472 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.117180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:46435 (local address 127.0.0.1:49366)
0807 08:32:07.117341 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.117397 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 08:32:07.117702 23316 catalog_manager.cc:4584] TS 5d5e7a321b1d448686ffa576c851f3bc (127.1.157.65:46435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c80440f11694dc9a1511d9b03e8c3e1: Network error: Client connection negotiation failed: client connection to 127.1.157.65:46435: connect: Connection refused (error 111)
I20260807 08:32:07.126323 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 23855
I20260807 08:32:07.134686 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 23288
I20260807 08:32:07.145970 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--webserver_interface=127.1.157.126
--webserver_port=36615
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:07.252571 24097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:07.252799 24097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:07.252852 24097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:07.256705 24097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:07.256754 24097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:07.256780 24097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:07.256800 24097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:07.256821 24097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:07.261811 24097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=36615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:07.262697 24097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:07.263588 24097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:07.271586 24108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.271586 24103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.271587 24104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.271947 24097 server_base.cc:1034] running on GCE node
I20260807 08:32:07.272436 24097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:07.273415 24097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:07.274562 24097 hybrid_clock.cc:648] HybridClock initialized: now 1786091527274545 us; error 27 us; skew 500 ppm
Aug 07 08:32:07 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:07.277055 24097 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:07.277860 24097 webserver.cc:460] Webserver started at http://127.1.157.126:36615/ using document root <none> and password file <none>
I20260807 08:32:07.278379 24097 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:07.278447 24097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:07.282014 24097 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:07.284101 24122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.285236 24097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 08:32:07.285337 24097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c7bcc7dfe907453cb99df7c1d6a95237"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "9b1565d36c5ece677519963f5d816423"
server_key_iv: "be1815eb4b8891f14f63b0d99c9256b9"
server_key_version: "encryptionkey@0"
I20260807 08:32:07.285858 24097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:07.308811 24097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:07.311475 24097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:07.311645 24097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:07.318302 24097 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:35097
I20260807 08:32:07.318578 24193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:35097 every 8 connection(s)
I20260807 08:32:07.319458 24097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:07.321933 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 24097
I20260807 08:32:07.322427 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:46435
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=46815
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:07.324065 24194 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:32:07.325685 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap starting.
I20260807 08:32:07.343039 24194 log.cc:824] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:07.353893 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:07.354219 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237: Bootstrap complete.
I20260807 08:32:07.358074 24194 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:07.358481 24194 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Initialized, Role: FOLLOWER
I20260807 08:32:07.358958 24194 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:07.359074 24194 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:07.359138 24194 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:07.359229 24194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:32:07.360977 24194 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:07.361295 24194 leader_election.cc:302] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bcc7dfe907453cb99df7c1d6a95237; no voters:
I20260807 08:32:07.361594 24194 leader_election.cc:288] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:32:07.361686 24198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:32:07.361871 24198 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [term 3 LEADER]: Becoming Leader. State: Replica: c7bcc7dfe907453cb99df7c1d6a95237, State: Running, Role: LEADER
I20260807 08:32:07.362205 24198 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } }
I20260807 08:32:07.362787 24194 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:07.363540 24199 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:07.363674 24199 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:07.363994 24200 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcc7dfe907453cb99df7c1d6a95237. Latest consensus state: current_term: 3 leader_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcc7dfe907453cb99df7c1d6a95237" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 35097 } } }
I20260807 08:32:07.364154 24200 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcc7dfe907453cb99df7c1d6a95237 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:07.364470 24210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:07.369496 24210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:07.370102 24210 catalog_manager.cc:1259] Loaded cluster ID: 553b6115430f4e13bee4f8bff8a70132
I20260807 08:32:07.370162 24210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 08:32:07.371270 24210 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1fbe1e29ad google::LogMessage::Fail() at ??:0
@ 0x7f1fbe1e588a google::LogMessage::SendToLog() at ??:0
@ 0x7f1fbe1e24de google::LogMessage::Flush() at ??:0
@ 0x7f1fbe1e3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1fc8b21fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1fc8b22d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1fc8b1f579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1fc8b63640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fc04343d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1fc0434c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1fc04363f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fc0425512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1fc1a696db start_thread at ??:0
@ 0x7f1fbcce061f clone at ??:0
W20260807 08:32:07.456418 24196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:07.456640 24196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:07.456678 24196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:07.460582 24196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:07.460628 24196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:07.460682 24196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:32:07.465670 24196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:46435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=46815
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:07.466615 24196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:07.467553 24196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:07.476836 24255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.476850 24258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.476853 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.477320 24196 server_base.cc:1034] running on GCE node
I20260807 08:32:07.477681 24196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:07.478245 24196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:07.479466 24196 hybrid_clock.cc:648] HybridClock initialized: now 1786091527479438 us; error 43 us; skew 500 ppm
Aug 07 08:32:07 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:07.481986 24196 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:32:07.482872 24196 webserver.cc:460] Webserver started at http://127.1.157.65:46815/ using document root <none> and password file <none>
I20260807 08:32:07.483378 24196 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:07.483436 24196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:07.487006 24196 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:07.488970 24278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.490106 24196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:07.490207 24196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "5d5e7a321b1d448686ffa576c851f3bc"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "f776c383eded33e56ab082624911a19a"
server_key_iv: "e8bf8bc03d308a12c347f685e43141af"
server_key_version: "encryptionkey@0"
I20260807 08:32:07.490748 24196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:07.509444 24196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:07.512285 24196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:07.512444 24196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:07.512985 24196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:07.514017 24287 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:32:07.517784 24196 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:32:07.517838 24196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 08:32:07.517881 24196 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:32:07.519490 24196 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:32:07.519547 24196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:07.519825 24287 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap starting.
I20260807 08:32:07.529696 24196 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:46435
I20260807 08:32:07.529830 24454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:46435 every 8 connection(s)
I20260807 08:32:07.530881 24196 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:32:07.539513 24287 log.cc:824] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:07.539872 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 24196
I20260807 08:32:07.540254 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:33657
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=39097
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:07.546761 24287 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:07.547137 24287 tablet_bootstrap.cc:492] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Bootstrap complete.
I20260807 08:32:07.548467 24287 ts_tablet_manager.cc:1397] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent bootstrapping tablet: real 0.029s user 0.026s sys 0.000s
I20260807 08:32:07.551654 24287 raft_consensus.cc:348] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:07.552209 24287 raft_consensus.cc:729] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5e7a321b1d448686ffa576c851f3bc, State: Initialized, Role: FOLLOWER
I20260807 08:32:07.552734 24287 consensus_queue.cc:260] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:07.553803 24287 ts_tablet_manager.cc:1428] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260807 08:32:07.570456 24460 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:32:07.532967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:49153)
0807 08:32:07.533538 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.533608 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:07.534716 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:07.569524 (+ 34808us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":157,"threads_started":1}
W20260807 08:32:07.571004 24455 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:35097 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:35097: Client connection negotiation failed: client connection to 127.1.157.126:35097: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:32:07.571709 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.571467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:54647)
0807 08:32:07.571583 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.571655 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:32:07.572263 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.572094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:39295)
0807 08:32:07.572180 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.572218 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 08:32:07.572431 24455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:32:07.656886 24465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:07.657097 24465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:07.657136 24465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:07.660921 24465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:07.660965 24465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:07.661017 24465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:32:07.665834 24465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:33657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=39097
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:07.666659 24465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:07.667523 24465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:07.676250 24474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.676322 24473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.676376 24465 server_base.cc:1034] running on GCE node
W20260807 08:32:07.676302 24476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.676807 24465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:07.677385 24465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:07.678519 24465 hybrid_clock.cc:648] HybridClock initialized: now 1786091527678496 us; error 35 us; skew 500 ppm
Aug 07 08:32:07 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:07.681140 24465 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:32:07.681967 24465 webserver.cc:460] Webserver started at http://127.1.157.66:39097/ using document root <none> and password file <none>
I20260807 08:32:07.682514 24465 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:07.682575 24465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:07.686142 24465 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:07.688198 24492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.689320 24465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:07.689426 24465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "41c639be52ec4ade91ea1e6d92e1e534"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "aa262559fbeed421468b9cdedfffa147"
server_key_iv: "37df593af54417f8ae5ed05dd83bc316"
server_key_version: "encryptionkey@0"
I20260807 08:32:07.689994 24465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:07.705513 24465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:07.708344 24465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:07.708513 24465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:07.709106 24465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:07.709920 24465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:07.709967 24465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.710018 24465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:07.710043 24465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.718729 24465 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:33657
I20260807 08:32:07.718784 24658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:33657 every 8 connection(s)
I20260807 08:32:07.719704 24465 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:32:07.720677 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 24465
I20260807 08:32:07.721055 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:35205
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:35097
--webserver_port=38757
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:07.723920 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.721966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:60757)
0807 08:32:07.722509 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.722987 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
W20260807 08:32:07.724352 24659 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:35097 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:35097: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
I20260807 08:32:07.724920 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.724703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:56815)
0807 08:32:07.724816 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.724864 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 08:32:07.725498 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.725313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:33073)
0807 08:32:07.725407 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.725449 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:32:07.725643 24659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:32:07.830000 24662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:07.830220 24662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:07.830276 24662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:07.834111 24662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:07.834163 24662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:07.834237 24662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:32:07.839154 24662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:35205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=38757
--tserver_master_addrs=127.1.157.126:35097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:07.840035 24662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:07.840945 24662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:07.849908 24670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.849978 24662 server_base.cc:1034] running on GCE node
W20260807 08:32:07.849929 24673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:07.849913 24671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:07.850438 24662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:07.850972 24662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:07.852119 24662 hybrid_clock.cc:648] HybridClock initialized: now 1786091527852099 us; error 28 us; skew 500 ppm
Aug 07 08:32:07 dist-test-slave-6zbq krb5kdc[22441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:07.857518 24662 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:32:07.858314 24662 webserver.cc:460] Webserver started at http://127.1.157.67:38757/ using document root <none> and password file <none>
I20260807 08:32:07.858836 24662 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:07.858894 24662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:07.862417 24662 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 08:32:07.864280 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.865383 24662 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:07.865471 24662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "bcfed4c5544f4c7092af69e52930e383"
format_stamp: "Formatted at 2026-08-07 08:32:05 on dist-test-slave-6zbq"
server_key: "b63326f14c45e857d2d2c175e8e86ad0"
server_key_iv: "5bd440542c704db7269ae3867ea14508"
server_key_version: "encryptionkey@0"
I20260807 08:32:07.865950 24662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:07.902052 24662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:07.905023 24662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:07.905164 24662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:07.905696 24662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:07.906476 24662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:07.906522 24662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.906559 24662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:07.906579 24662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:07.914862 24662 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:35205
I20260807 08:32:07.914896 24845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:35205 every 8 connection(s)
I20260807 08:32:07.915784 24662 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:32:07.918851 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 24662
I20260807 08:32:07.919168 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.917701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:45687)
0807 08:32:07.918321 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.918468 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":107,"threads_started":1}
W20260807 08:32:07.919595 24847 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:35097 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:35097: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
I20260807 08:32:07.920147 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.919955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:49651)
0807 08:32:07.920050 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.920094 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 08:32:07.920680 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:07.920494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:52781)
0807 08:32:07.920587 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:07.920630 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:32:07.920830 24847 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:32:08.573303 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.572918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:47981)
0807 08:32:08.573143 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:08.573204 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260807 08:32:08.723901 24852 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:08.724054 24852 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:08.724980 24852 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657), bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
I20260807 08:32:08.726500 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.726102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:46727)
0807 08:32:08.726322 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:08.726397 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":74,"threads_started":1}
Aug 07 08:32:08 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.67@KRBTEST.COM
Aug 07 08:32:08 dist-test-slave-6zbq krb5kdc[22441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.66@KRBTEST.COM
I20260807 08:32:08.737426 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.725394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35205 (local address 127.1.157.65:38141)
0807 08:32:08.725619 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:08.725630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:08.725731 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:08.727424 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:08.727427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:08.727935 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:08.728402 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:08.728415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.729612 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.729615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:08.730589 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:08.730599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.731238 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.731242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:08.731459 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:08.732029 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:08.732044 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:08.733429 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:08.735306 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:08.735311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:08.735324 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:08.735552 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:08.735863 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:08.735866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:08.735868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:08.735989 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:08.736548 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:08.736554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:08.736761 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 08:32:08.736914 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 08:32:08.737208 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":75,"spinlock_wait_cycles":47744,"thread_start_us":67,"threads_started":1}
I20260807 08:32:08.737452 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.725554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:38141 (local address 127.1.157.67:35205)
0807 08:32:08.725849 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:08.725859 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:08.725893 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:32:08.726675 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:08.726684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:08.727054 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:08.727353 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:08.728557 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.729484 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.730695 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.731128 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.731268 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:08.733610 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:08.733614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:08.733624 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:08.733650 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:08.735200 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:08.735657 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:08.735661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:08.735665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:08.735776 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:08.736073 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:08.736076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:08.736078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:08.736439 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:08.736551 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:08.737040 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 08:32:08.737299 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260807 08:32:08.737461 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.725285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:34101 (local address 127.1.157.66:33657)
0807 08:32:08.725909 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:08.725923 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:08.725957 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:32:08.726679 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:08.726685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:08.727050 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:08.727347 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:08.728552 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.729480 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.730696 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.731128 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.731279 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:08.733914 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:08.733921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:08.733934 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:08.733964 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:08.735532 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:08.735917 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:08.735922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:08.735927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:08.736033 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:08.736276 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:08.736279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:08.736281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:08.736620 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:08.736730 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:08.737043 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 08:32:08.737280 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":55,"threads_started":1}
I20260807 08:32:08.737447 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.725147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:33657 (local address 127.1.157.65:34101)
0807 08:32:08.725265 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:08.725279 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:08.725368 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:08.727420 (+ 2052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:08.727426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:08.727936 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:08.728403 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:08.728416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.729621 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.729624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:08.730588 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:08.730598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:08.731246 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:08.731250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:08.731461 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:08.732082 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:08.732093 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:08.733747 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:08.735633 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:08.735637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:08.735639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:08.735822 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:08.736119 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:08.736122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:08.736124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:08.736200 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:08.736718 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:08.736721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:08.736818 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 08:32:08.736915 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:32:08.737280 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":264,"spinlock_wait_cycles":6400}
W20260807 08:32:08.739868 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
W20260807 08:32:08.740049 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:08.740104 24281 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc; no voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383
I20260807 08:32:08.740329 24852 raft_consensus.cc:2726] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:32:08.921677 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:08.921301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:58969)
0807 08:32:08.921540 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:08.921590 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260807 08:32:09.574304 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:09.573927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:56641)
0807 08:32:09.574152 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:09.574204 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260807 08:32:09.727579 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:09.727215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:38605)
0807 08:32:09.727443 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:09.727497 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260807 08:32:09.922685 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:09.922251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:57019)
0807 08:32:09.922514 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:09.922569 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260807 08:32:10.374580 24867 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:10.374719 24867 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:10.375093 24867 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657), bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
W20260807 08:32:10.375905 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
W20260807 08:32:10.376030 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:10.376070 24281 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc; no voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383
I20260807 08:32:10.376224 24867 raft_consensus.cc:2726] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:32:10.575340 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:10.574975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:42361)
0807 08:32:10.575198 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:10.575248 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260807 08:32:10.728533 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:10.728151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:42393)
0807 08:32:10.728378 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:10.728438 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260807 08:32:10.923655 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:10.923299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:55283)
0807 08:32:10.923518 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:10.923567 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260807 08:32:11.576366 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:11.575917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:50603)
0807 08:32:11.576182 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:11.576245 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260807 08:32:11.729617 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:11.729212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:37899)
0807 08:32:11.729465 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:11.729521 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260807 08:32:11.924671 24876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:11.924291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:44521)
0807 08:32:11.924524 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:11.924577 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":75,"threads_started":1}
I20260807 08:32:12.274621 24877 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:12.274757 24877 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:12.275162 24877 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657), bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
W20260807 08:32:12.275897 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
W20260807 08:32:12.276031 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:12.276083 24281 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc; no voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383
I20260807 08:32:12.276237 24877 raft_consensus.cc:2726] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:32:12.577636 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:12.577165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:40117)
0807 08:32:12.577456 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:12.577518 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260807 08:32:12.730724 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:12.730290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:52823)
0807 08:32:12.730573 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:12.730634 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":117,"threads_started":1}
I20260807 08:32:12.925618 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:12.925260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:48185)
0807 08:32:12.925487 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:12.925537 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260807 08:32:13.578706 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:13.578334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:51287)
0807 08:32:13.578569 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:13.578620 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260807 08:32:13.731755 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:13.731376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:42625)
0807 08:32:13.731619 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:13.731672 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":70,"threads_started":1}
I20260807 08:32:13.926656 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:13.926245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:55103)
0807 08:32:13.926499 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:13.926552 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
I20260807 08:32:14.579751 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:14.579347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:40655)
0807 08:32:14.579601 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:14.579656 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260807 08:32:14.732759 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:14.732348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:53229)
0807 08:32:14.732615 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:14.732665 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
I20260807 08:32:14.927641 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:14.927253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:58699)
0807 08:32:14.927512 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:14.927563 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":63,"threads_started":1}
I20260807 08:32:15.580789 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:15.580395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:53227)
0807 08:32:15.580642 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:15.580694 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260807 08:32:15.733922 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:15.733493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:44505)
0807 08:32:15.733752 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:15.733826 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":95,"threads_started":1}
I20260807 08:32:15.928746 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:15.928283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:46693)
0807 08:32:15.928586 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:15.928642 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260807 08:32:15.989462 24903 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:15.989589 24903 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:15.989924 24903 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657), bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
W20260807 08:32:15.990715 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
W20260807 08:32:15.990834 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:15.990872 24281 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc; no voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383
I20260807 08:32:15.990999 24903 raft_consensus.cc:2726] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:32:16.581873 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:16.581495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:41723)
0807 08:32:16.581727 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:16.581780 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260807 08:32:16.734970 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:16.734588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:54091)
0807 08:32:16.734831 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:16.734883 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":69,"threads_started":1}
I20260807 08:32:16.929816 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:16.929451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:41445)
0807 08:32:16.929678 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:16.929731 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
I20260807 08:32:17.582990 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:17.582624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:42151)
0807 08:32:17.582847 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:17.582900 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260807 08:32:17.735945 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:17.735596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:55883)
0807 08:32:17.735808 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:17.735858 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260807 08:32:17.858134 24913 raft_consensus.cc:482] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:17.858275 24913 raft_consensus.cc:504] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41c639be52ec4ade91ea1e6d92e1e534" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 33657 } } peers { permanent_uuid: "bcfed4c5544f4c7092af69e52930e383" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35205 } } peers { permanent_uuid: "5d5e7a321b1d448686ffa576c851f3bc" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 46435 } }
I20260807 08:32:17.858691 24913 leader_election.cc:288] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657), bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205)
W20260807 08:32:17.859326 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41c639be52ec4ade91ea1e6d92e1e534 (127.1.157.66:33657): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
W20260807 08:32:17.859452 24281 leader_election.cc:341] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcfed4c5544f4c7092af69e52930e383 (127.1.157.67:35205): Not found: Tablet not found: 7c80440f11694dc9a1511d9b03e8c3e1
I20260807 08:32:17.859503 24281 leader_election.cc:302] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d5e7a321b1d448686ffa576c851f3bc; no voters: 41c639be52ec4ade91ea1e6d92e1e534, bcfed4c5544f4c7092af69e52930e383
I20260807 08:32:17.859659 24913 raft_consensus.cc:2726] T 7c80440f11694dc9a1511d9b03e8c3e1 P 5d5e7a321b1d448686ffa576c851f3bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:32:17.930815 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:17.930446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:59837)
0807 08:32:17.930682 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:17.930733 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260807 08:32:18.583983 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:18.583611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:37675)
0807 08:32:18.583837 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:18.583889 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":73,"threads_started":1}
I20260807 08:32:18.736893 24918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:18.736519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:44835)
0807 08:32:18.736756 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:18.736808 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
I20260807 08:32:18.931770 24920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:18.931416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:38775)
0807 08:32:18.931625 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:18.931675 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
I20260807 08:32:19.585109 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:19.584704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:60763)
0807 08:32:19.584962 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:19.585015 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260807 08:32:19.737886 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:19.737526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:44733)
0807 08:32:19.737748 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:19.737797 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":70,"threads_started":1}
I20260807 08:32:19.932735 24924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:19.932378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:36803)
0807 08:32:19.932599 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:19.932647 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260807 08:32:20.586140 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:20.585748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:41251)
0807 08:32:20.585998 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:20.586049 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260807 08:32:20.738926 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:20.738526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:60067)
0807 08:32:20.738787 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:20.738841 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
I20260807 08:32:20.933761 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:20.933384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:50387)
0807 08:32:20.933617 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:20.933670 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260807 08:32:21.587229 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:21.586832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:56021)
0807 08:32:21.587072 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:21.587122 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260807 08:32:21.739982 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:21.739581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:49041)
0807 08:32:21.739842 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:21.739892 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":114,"threads_started":1}
I20260807 08:32:21.935228 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:21.934639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:36963)
0807 08:32:21.935023 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:21.935101 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":136,"threads_started":1}
I20260807 08:32:22.588393 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:22.588011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.65:41461)
0807 08:32:22.588251 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:22.588300 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260807 08:32:22.741046 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:22.740661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.66:36361)
0807 08:32:22.740910 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:22.740963 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260807 08:32:22.928668 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 24196
I20260807 08:32:22.937091 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:22.936494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:35097 (local address 127.1.157.67:36535)
0807 08:32:22.936921 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:22.936980 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":189,"threads_started":1}
I20260807 08:32:22.954140 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 24465
I20260807 08:32:22.964686 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 24662
2026-08-07T08:32:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19098 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:32:23 dist-test-slave-6zbq krb5kdc[24948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:32:23 dist-test-slave-6zbq krb5kdc[24948](info): set up 2 sockets
Aug 07 08:32:23 dist-test-slave-6zbq krb5kdc[24948](info): commencing operation
krb5kdc: starting...
W20260807 08:32:24.088375 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:32:24.202615 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:24.310416 24961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:24.310659 24961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:24.310714 24961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:24.314637 24961 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:32:24.314679 24961 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:32:24.314702 24961 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:32:24.314718 24961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:24.314742 24961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:24.314775 24961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:24.314800 24961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:24.314834 24961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:24.319890 24961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:24.320887 24961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:24.321893 24961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:24.330371 24967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.330364 24969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.330359 24966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:24.330497 24961 server_base.cc:1034] running on GCE node
I20260807 08:32:24.331236 24961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:24.332197 24961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:24.333365 24961 hybrid_clock.cc:648] HybridClock initialized: now 1786091544333349 us; error 28 us; skew 500 ppm
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:24.335952 24961 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:24.336738 24961 webserver.cc:460] Webserver started at http://127.1.157.126:37553/ using document root <none> and password file <none>
I20260807 08:32:24.337255 24961 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:24.337322 24961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:24.337508 24961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:24.339490 24961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "ad51bb7ec25e4ef6b9316baa6144da21"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "083b925400bc4b66b602ddce475d42a1"
server_key_iv: "733c7129382f7267c60c9fab579044f4"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.340021 24961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "ad51bb7ec25e4ef6b9316baa6144da21"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "083b925400bc4b66b602ddce475d42a1"
server_key_iv: "733c7129382f7267c60c9fab579044f4"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.343429 24961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:32:24.345537 24980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.346652 24961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:32:24.346764 24961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "ad51bb7ec25e4ef6b9316baa6144da21"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "083b925400bc4b66b602ddce475d42a1"
server_key_iv: "733c7129382f7267c60c9fab579044f4"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.346849 24961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:24.362387 24961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:24.365281 24961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:24.365449 24961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:24.371973 24961 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43983
I20260807 08:32:24.372211 25061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43983 every 8 connection(s)
I20260807 08:32:24.372849 24961 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:24.375484 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:24.377849 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 24961
I20260807 08:32:24.377959 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:32:24.378204 1653 external_mini_cluster.cc:1428] Setting key 2211b87e2a96614c9c28f7e46d77688b
I20260807 08:32:24.381249 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap starting.
I20260807 08:32:24.383769 25063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:24.384445 25063 log.cc:824] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:24.386199 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: No bootstrap required, opened a new log
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:24.389304 25063 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:24.389498 25063 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:24.389532 25063 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Initialized, Role: FOLLOWER
I20260807 08:32:24.389943 25063 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:24.390102 25063 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:24.390168 25063 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:24.390269 25063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:24.391237 25063 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:24.391615 25063 leader_election.cc:302] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad51bb7ec25e4ef6b9316baa6144da21; no voters:
I20260807 08:32:24.391963 25063 leader_election.cc:288] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:32:24.392053 25072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:24.392218 25072 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 LEADER]: Becoming Leader. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Running, Role: LEADER
I20260807 08:32:24.392565 25072 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:24.392735 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:24.379481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.1.157.126:43983)
0807 08:32:24.379892 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:24.379910 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:24.379951 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:32:24.380740 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:24.380762 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:24.381163 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:24.381500 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:24.382139 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.383635 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.384483 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.385041 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.385195 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:24.387708 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:24.387714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:24.387725 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:24.387751 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:24.389821 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.390305 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.390310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.390314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.390480 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.390771 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.390773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.390775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.391143 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:24.391301 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:24.391583 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 08:32:24.391851 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260807 08:32:24.393164 25063 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:24.394081 25073 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:32:24.395344 25073 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:24.394950 25094 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad51bb7ec25e4ef6b9316baa6144da21. Latest consensus state: current_term: 1 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:32:24.395943 25094 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
W20260807 08:32:24.398008 25101 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:32:24.398073 25101 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:32:24.398131 25103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:24.398898 25103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:24.404197 25103 catalog_manager.cc:1347] Generated new cluster ID: c7b4608acb6e4e06baa58526afdab13f
I20260807 08:32:24.404258 25103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:24.411855 25103 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:32:24.412971 25103 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:24.421770 25103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Generated new TSK 0
I20260807 08:32:24.422477 25103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:32:24.454849 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:24.560923 25111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:24.561154 25111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:24.561199 25111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:24.565011 25111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:24.565058 25111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:24.565133 25111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:32:24.569963 25111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:24.570837 25111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:24.571707 25111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:24.580338 25116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.580426 25117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:24.580456 25111 server_base.cc:1034] running on GCE node
W20260807 08:32:24.580329 25119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:24.580927 25111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:24.581462 25111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:24.582623 25111 hybrid_clock.cc:648] HybridClock initialized: now 1786091544582593 us; error 30 us; skew 500 ppm
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:24.585069 25111 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:32:24.585811 25111 webserver.cc:460] Webserver started at http://127.1.157.65:40729/ using document root <none> and password file <none>
I20260807 08:32:24.586282 25111 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:24.586366 25111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:24.586556 25111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:24.588374 25111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "0e74b23bf2204ec79a25b4169a552c14"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "8e6264c8ae4c459ba478c55738ce168f"
server_key_iv: "7606633326ff7de22e223fa1e9c19a19"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.588884 25111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "0e74b23bf2204ec79a25b4169a552c14"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "8e6264c8ae4c459ba478c55738ce168f"
server_key_iv: "7606633326ff7de22e223fa1e9c19a19"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.592213 25111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:32:24.594460 25128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.595566 25111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:32:24.595678 25111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "0e74b23bf2204ec79a25b4169a552c14"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "8e6264c8ae4c459ba478c55738ce168f"
server_key_iv: "7606633326ff7de22e223fa1e9c19a19"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.595763 25111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:24.615024 25111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:24.617781 25111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:24.617988 25111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:24.618631 25111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:24.619539 25111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:24.619596 25111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.619653 25111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:24.619714 25111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.627635 25111 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:37649
I20260807 08:32:24.627676 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:37649 every 8 connection(s)
I20260807 08:32:24.628553 25111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:32:24.630317 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 25111
I20260807 08:32:24.630460 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:32:24.630719 1653 external_mini_cluster.cc:1428] Setting key a4484ee284666fb18e52ef7d12e43ca5
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:32:24.641014 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:24.630175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:49373 (local address 127.1.157.126:43983)
0807 08:32:24.630310 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:24.630314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:24.631323 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 08:32:24.632174 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:24.632177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:24.632229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:24.632297 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:24.633308 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.633959 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.635089 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.635419 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.635519 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:24.637679 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:24.637683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:24.637685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:24.637714 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:24.639213 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.639685 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.639689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.639690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.639778 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.640053 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.640056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.640058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.640292 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:24.640393 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:24.640809 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 08:32:24.640903 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:32:24.641716 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:24.630541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:49373)
0807 08:32:24.631190 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:24.631237 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:24.631989 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:24.632443 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:24.632449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:24.632764 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:24.633173 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:24.633186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.634086 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.634089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:24.634967 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:24.634978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.635555 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.635558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:24.635696 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:24.636159 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:24.636174 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:24.637523 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:24.639336 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:24.639341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:24.639351 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:24.639583 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:24.639861 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:24.639863 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:24.639864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:24.639976 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:24.640400 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:24.640405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:24.640610 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 08:32:24.640751 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 08:32:24.640969 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
I20260807 08:32:24.642884 25296 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:24.643106 25296 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:24.643524 25296 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:24.644934 25006 ts_manager.cc:194] Registered new tserver with Master: 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649)
I20260807 08:32:24.646301 25006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:49373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:32:24.656275 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:24.759927 25302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:24.760138 25302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:24.760183 25302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:24.764045 25302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:24.764088 25302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:24.764161 25302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:32:24.768988 25302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:24.769817 25302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:24.770705 25302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:24.778945 25309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.778978 25311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.779002 25308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:24.779269 25302 server_base.cc:1034] running on GCE node
I20260807 08:32:24.779599 25302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:24.780112 25302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:24.781268 25302 hybrid_clock.cc:648] HybridClock initialized: now 1786091544781245 us; error 34 us; skew 500 ppm
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:24.783761 25302 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:32:24.784484 25302 webserver.cc:460] Webserver started at http://127.1.157.66:41039/ using document root <none> and password file <none>
I20260807 08:32:24.784942 25302 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:24.784999 25302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:24.785187 25302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:24.786999 25302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "b970f2c8cc794782955cea7e38004a12"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "b75585c93c348c5c4a2630cd88e0bbd1"
server_key_iv: "0299a5c4fefcaa57c9a0550ef24b8242"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.787503 25302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "b970f2c8cc794782955cea7e38004a12"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "b75585c93c348c5c4a2630cd88e0bbd1"
server_key_iv: "0299a5c4fefcaa57c9a0550ef24b8242"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.790751 25302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:32:24.792858 25323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.793958 25302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:24.794073 25302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "b970f2c8cc794782955cea7e38004a12"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "b75585c93c348c5c4a2630cd88e0bbd1"
server_key_iv: "0299a5c4fefcaa57c9a0550ef24b8242"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.794173 25302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:24.810286 25302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:24.812952 25302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:24.813081 25302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:24.813527 25302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:24.814204 25302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:24.814248 25302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.814282 25302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:24.814302 25302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.822009 25302 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37005
I20260807 08:32:24.822039 25494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37005 every 8 connection(s)
I20260807 08:32:24.822741 25302 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:24.831249 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 25302
I20260807 08:32:24.831368 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:32:24.831663 1653 external_mini_cluster.cc:1428] Setting key 9d7fafe3161ea676600c1ae7a2ca91fb
I20260807 08:32:24.834115 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:24.824235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:49775 (local address 127.1.157.126:43983)
0807 08:32:24.824361 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:24.824365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:24.825213 (+ 848us) server_negotiation.cc:407] Connection header received
0807 08:32:24.825941 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:24.825943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:24.825988 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:24.826053 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:24.827064 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.827562 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.828553 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.828756 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.828821 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:24.830830 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:24.830833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:24.830834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:24.830860 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:24.832207 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.832708 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.832711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.832712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.832787 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:24.833138 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:24.833141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:24.833143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:24.833353 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:24.833452 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:24.833856 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 08:32:24.833968 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:32:24.834723 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:24.824503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:49775)
0807 08:32:24.825089 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:24.825124 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:24.825788 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:24.826185 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:24.826191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:24.826538 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:24.826955 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:24.826967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.827659 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.827662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:24.828468 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:24.828477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:24.828852 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:24.828855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:24.828979 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:24.829375 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:24.829390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:24.830676 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:24.832304 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:24.832308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:24.832318 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:24.832628 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:24.832875 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:24.832877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:24.832879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:24.833046 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:24.833442 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:24.833446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:24.833646 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 08:32:24.833780 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 08:32:24.833985 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":112,"threads_started":1}
I20260807 08:32:24.835649 25495 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:24.835839 25495 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:24.836238 25495 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:24.837093 25006 ts_manager.cc:194] Registered new tserver with Master: b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
I20260807 08:32:24.837599 25006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:49775
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:32:24.855077 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:24.959257 25500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:24.959479 25500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:24.959522 25500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:24.963414 25500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:24.963462 25500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:24.963531 25500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:32:24.968446 25500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:24.969352 25500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:24.970276 25500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:24.978861 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.978852 25511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:24.978854 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:24.978974 25500 server_base.cc:1034] running on GCE node
I20260807 08:32:24.979475 25500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:24.980006 25500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:24.981171 25500 hybrid_clock.cc:648] HybridClock initialized: now 1786091544981150 us; error 31 us; skew 500 ppm
Aug 07 08:32:24 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:24.983695 25500 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:32:24.984483 25500 webserver.cc:460] Webserver started at http://127.1.157.67:41551/ using document root <none> and password file <none>
I20260807 08:32:24.984968 25500 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:24.985039 25500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:24.985237 25500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:32:24.987166 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "f8e9d323e00d484f867ad17d4ff94bea"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "14b39b5fe9955bb6b4adecf3eab3f589"
server_key_iv: "58f2ac345b9f8d102c47bafa9e826304"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.987676 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "f8e9d323e00d484f867ad17d4ff94bea"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "14b39b5fe9955bb6b4adecf3eab3f589"
server_key_iv: "58f2ac345b9f8d102c47bafa9e826304"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.991020 25500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:32:24.993207 25520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:24.994402 25500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:24.994505 25500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "f8e9d323e00d484f867ad17d4ff94bea"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "14b39b5fe9955bb6b4adecf3eab3f589"
server_key_iv: "58f2ac345b9f8d102c47bafa9e826304"
server_key_version: "encryptionkey@0"
I20260807 08:32:24.994593 25500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:25.007766 25500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:25.010169 25500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:25.010322 25500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:25.010866 25500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:25.011591 25500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:25.011641 25500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:25.011691 25500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:25.011718 25500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:25.019630 25500 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:42049
I20260807 08:32:25.019656 25685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:42049 every 8 connection(s)
I20260807 08:32:25.020423 25500 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:32:25 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:25.030071 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 25500
I20260807 08:32:25.030189 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:32:25.030515 1653 external_mini_cluster.cc:1428] Setting key 3e99b175c3bf719c9e87c6d9c099dfa3
I20260807 08:32:25.032579 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.022029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:35569 (local address 127.1.157.126:43983)
0807 08:32:25.022186 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.022190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.023124 (+ 934us) server_negotiation.cc:407] Connection header received
0807 08:32:25.023971 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.023974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.024031 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.024109 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:25.025189 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.025710 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.026813 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.027027 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.027114 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.029277 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:25.029280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:25.029282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:25.029304 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:25.030621 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:25.031082 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:25.031085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:25.031086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:25.031188 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:25.031544 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:25.031547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:25.031549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:25.031769 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:25.031854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.032303 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.032430 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 08:32:25.033249 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.022332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:35569)
0807 08:32:25.022981 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.023019 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.023779 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.024283 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.024290 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.024613 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:25.025043 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.025055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.025827 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.025830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.026692 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.026702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.027168 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.027172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.027329 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.027776 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:25.027790 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:25.029128 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:25.030725 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:25.030730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:25.030739 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:25.030978 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:25.031278 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:25.031282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:25.031284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:25.031455 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:25.031866 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:25.031872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:25.032084 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.032240 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.032517 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":107,"threads_started":1}
I20260807 08:32:25.034204 25687 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:25.034430 25687 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:25.034838 25687 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:25.035734 25006 ts_manager.cc:194] Registered new tserver with Master: f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049)
I20260807 08:32:25.036235 25006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:35569
I20260807 08:32:25.043926 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:32:25.051720 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.045660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.1.157.126:43983)
0807 08:32:25.045750 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.045753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.045926 (+ 173us) server_negotiation.cc:407] Connection header received
0807 08:32:25.046033 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.046035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.046069 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.046136 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:25.046760 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.047231 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.047943 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.048163 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.048237 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.049302 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:25.049305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:25.049306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:25.049327 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:25.050484 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:25.050841 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:25.050844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:25.050846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:25.050925 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:25.051115 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:25.051117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:25.051118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:25.051299 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:25.051398 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.051540 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.051623 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:32:25.053936 25006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:32:25.056807 25006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:32:25.069753 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.0.0.1:44468)
0807 08:32:25.064871 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.064914 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.065038 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.065781 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.065784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.065798 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.065905 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.065912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.067040 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.067043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.068430 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.068442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069264 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.069372 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069406 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.069503 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.069611 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":88,"thread_start_us":91,"threads_started":1}
I20260807 08:32:25.070050 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.1.157.67:42049)
0807 08:32:25.065299 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.065319 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.065337 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:32:25.065390 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.065394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.065528 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.066003 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.066108 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.067100 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.068582 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069359 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069503 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069629 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.069827 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.069919 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":124,"threads_started":1}
I20260807 08:32:25.070254 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.0.0.1:52174)
0807 08:32:25.064953 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.064964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.065037 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.065684 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.065690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.065711 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.065858 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.065866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.067227 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.067230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.068436 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.068442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069518 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.069603 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069625 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.069692 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.069750 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":46,"thread_start_us":90,"threads_started":1}
I20260807 08:32:25.070248 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.1.157.65:37649)
0807 08:32:25.065373 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.065386 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.065690 (+ 304us) server_negotiation.cc:407] Connection header received
0807 08:32:25.065765 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.065769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.065902 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.066021 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.066444 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.067449 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.068707 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069479 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069564 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069690 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.070014 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.070106 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":129,"threads_started":1}
I20260807 08:32:25.070289 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.1.157.66:37005)
0807 08:32:25.064697 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.064708 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.065171 (+ 463us) server_negotiation.cc:407] Connection header received
0807 08:32:25.065274 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.065280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.065472 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.065620 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.066027 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.066914 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.068555 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069150 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069260 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069402 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.070055 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.070156 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":100,"threads_started":1}
I20260807 08:32:25.070551 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.064473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:37649 (local address 127.0.0.1:60826)
0807 08:32:25.065502 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.065514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.065597 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.066195 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.066198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.066216 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.066305 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.066311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.067561 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.067564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.068586 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.068597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.069703 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.069707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.069781 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.069803 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.069873 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.069937 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260807 08:32:25.072700 25599 tablet_service.cc:1467] Processing CreateTablet for tablet e398cbf86e7944018209e7d5a83c40ba (DEFAULT_TABLE table=test-table [id=2ebff11a358b463b8be3097df7002ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:25.072690 25397 tablet_service.cc:1467] Processing CreateTablet for tablet e398cbf86e7944018209e7d5a83c40ba (DEFAULT_TABLE table=test-table [id=2ebff11a358b463b8be3097df7002ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:25.073534 25397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e398cbf86e7944018209e7d5a83c40ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:25.073606 25599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e398cbf86e7944018209e7d5a83c40ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:25.074898 25197 tablet_service.cc:1467] Processing CreateTablet for tablet e398cbf86e7944018209e7d5a83c40ba (DEFAULT_TABLE table=test-table [id=2ebff11a358b463b8be3097df7002ad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:25.075783 25197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e398cbf86e7944018209e7d5a83c40ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:25.079156 25709 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Bootstrap starting.
I20260807 08:32:25.080827 25710 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Bootstrap starting.
I20260807 08:32:25.081892 25713 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap starting.
I20260807 08:32:25.082015 25709 tablet_bootstrap.cc:654] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:25.082928 25709 log.cc:824] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:25.083259 25710 tablet_bootstrap.cc:654] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:25.084012 25713 tablet_bootstrap.cc:654] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:25.084019 25710 log.cc:824] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:25.084771 25713 log.cc:824] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:25.085083 25709 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: No bootstrap required, opened a new log
I20260807 08:32:25.085260 25709 ts_tablet_manager.cc:1397] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:32:25.085656 25710 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: No bootstrap required, opened a new log
I20260807 08:32:25.085896 25710 ts_tablet_manager.cc:1397] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:32:25.086797 25713 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: No bootstrap required, opened a new log
I20260807 08:32:25.086977 25713 ts_tablet_manager.cc:1397] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:32:25.088600 25710 raft_consensus.cc:348] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.088814 25710 raft_consensus.cc:374] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:25.088891 25710 raft_consensus.cc:729] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e9d323e00d484f867ad17d4ff94bea, State: Initialized, Role: FOLLOWER
I20260807 08:32:25.088856 25709 raft_consensus.cc:348] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.089188 25709 raft_consensus.cc:374] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:25.089257 25709 raft_consensus.cc:729] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b970f2c8cc794782955cea7e38004a12, State: Initialized, Role: FOLLOWER
I20260807 08:32:25.089278 25710 consensus_queue.cc:260] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.089696 25709 consensus_queue.cc:260] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.090152 25710 ts_tablet_manager.cc:1428] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:32:25.090322 25495 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:25.090641 25709 ts_tablet_manager.cc:1428] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:32:25.090543 25713 raft_consensus.cc:348] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.090790 25713 raft_consensus.cc:374] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:25.090837 25713 raft_consensus.cc:729] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e74b23bf2204ec79a25b4169a552c14, State: Initialized, Role: FOLLOWER
I20260807 08:32:25.091194 25713 consensus_queue.cc:260] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.091378 25687 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:25.091847 25296 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:25.093092 25713 ts_tablet_manager.cc:1428] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 08:32:25.129485 25297 tablet.cc:2369] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:25.188051 25723 raft_consensus.cc:482] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:25.188220 25723 raft_consensus.cc:504] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.189160 25723 leader_election.cc:288] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649)
I20260807 08:32:25.192586 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.189325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.1.157.66:47959)
0807 08:32:25.189422 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.189437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.189518 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.189796 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.189799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.189816 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.189917 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.189922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.190668 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.190671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.191537 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.191547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.192198 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.192202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.192310 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.192332 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.192394 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.192455 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":51}
I20260807 08:32:25.192631 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.189494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:47959 (local address 127.1.157.67:42049)
0807 08:32:25.189623 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.189626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.189636 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:32:25.189674 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.189677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.189710 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.189769 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.190018 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.190546 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.191660 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.192076 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.192166 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.192212 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.192428 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.192502 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:32:25.192715 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.189366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:37649 (local address 127.1.157.66:41217)
0807 08:32:25.189676 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.189686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.189779 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.189979 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.189982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.189996 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.190102 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.190107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.190847 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.190850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.191627 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.191639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.192271 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.192274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.192363 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.192386 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.192461 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.192527 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260807 08:32:25.192718 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.189422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:41217 (local address 127.1.157.65:37649)
0807 08:32:25.189537 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.189541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.189763 (+ 222us) server_negotiation.cc:407] Connection header received
0807 08:32:25.189858 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.189861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.189902 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.189961 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.190211 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.190749 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.191765 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.192135 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.192244 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.192290 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.192505 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.192574 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:32:25.193296 25226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "b970f2c8cc794782955cea7e38004a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e74b23bf2204ec79a25b4169a552c14" is_pre_election: true
I20260807 08:32:25.193356 25629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "b970f2c8cc794782955cea7e38004a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e9d323e00d484f867ad17d4ff94bea" is_pre_election: true
I20260807 08:32:25.193543 25226 raft_consensus.cc:2454] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b970f2c8cc794782955cea7e38004a12 in term 0.
I20260807 08:32:25.193586 25629 raft_consensus.cc:2454] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b970f2c8cc794782955cea7e38004a12 in term 0.
I20260807 08:32:25.193642 25724 raft_consensus.cc:482] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:25.193719 25724 raft_consensus.cc:504] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.194018 25324 leader_election.cc:302] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14, b970f2c8cc794782955cea7e38004a12; no voters:
I20260807 08:32:25.194288 25723 raft_consensus.cc:2781] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:32:25.194411 25723 raft_consensus.cc:482] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:32:25.194489 25723 raft_consensus.cc:3037] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:25.194809 25724 leader_election.cc:288] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005), f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049)
I20260807 08:32:25.195923 25723 raft_consensus.cc:504] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.196395 25723 leader_election.cc:288] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [CANDIDATE]: Term 1 election: Requested vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649)
I20260807 08:32:25.196645 25629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "b970f2c8cc794782955cea7e38004a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e9d323e00d484f867ad17d4ff94bea"
I20260807 08:32:25.196765 25629 raft_consensus.cc:3037] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:25.196796 25226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "b970f2c8cc794782955cea7e38004a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e74b23bf2204ec79a25b4169a552c14"
I20260807 08:32:25.196898 25226 raft_consensus.cc:3037] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:25.197957 25629 raft_consensus.cc:2454] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b970f2c8cc794782955cea7e38004a12 in term 1.
I20260807 08:32:25.197957 25226 raft_consensus.cc:2454] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b970f2c8cc794782955cea7e38004a12 in term 1.
I20260807 08:32:25.198381 25326 leader_election.cc:302] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea; no voters:
I20260807 08:32:25.198580 25723 raft_consensus.cc:2781] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:25.198798 25723 raft_consensus.cc:686] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 LEADER]: Becoming Leader. State: Replica: b970f2c8cc794782955cea7e38004a12, State: Running, Role: LEADER
I20260807 08:32:25.199141 25723 consensus_queue.cc:237] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:25.199409 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.194960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.1.157.65:50335)
0807 08:32:25.195291 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.195304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.195382 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.195646 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.195648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.195663 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.195752 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.195758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.196610 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.196613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.198152 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.198161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.199003 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.199006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.199101 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.199126 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.199192 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.199256 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":79,"thread_start_us":61,"threads_started":1}
I20260807 08:32:25.199448 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.195045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.1.157.65:56127)
0807 08:32:25.195392 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.195403 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:25.195525 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:25.195770 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:25.195772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:25.195786 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:25.195893 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.195899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.196601 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.196604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.198066 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:25.198078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.199129 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.199132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:25.199208 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.199226 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:32:25.199289 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:32:25.199340 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":37,"thread_start_us":77,"threads_started":1}
I20260807 08:32:25.199572 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.195135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:56127 (local address 127.1.157.67:42049)
0807 08:32:25.195265 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.195269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.195468 (+ 199us) server_negotiation.cc:407] Connection header received
0807 08:32:25.195631 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.195634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.195676 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.195752 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.195978 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.196474 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.198230 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.198717 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.198786 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.198828 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.199416 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.199481 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 08:32:25.199815 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.195234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:50335 (local address 127.1.157.66:37005)
0807 08:32:25.195329 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.195333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.195346 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:32:25.195493 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.195496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.195537 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.195600 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:25.195884 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.196527 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.198287 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.198893 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.198993 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.199032 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.199683 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.199741 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:32:25.200344 25629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e9d323e00d484f867ad17d4ff94bea" is_pre_election: true
I20260807 08:32:25.200511 25629 raft_consensus.cc:2379] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e74b23bf2204ec79a25b4169a552c14 in current term 1: Already voted for candidate b970f2c8cc794782955cea7e38004a12 in this term.
I20260807 08:32:25.200541 25429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e398cbf86e7944018209e7d5a83c40ba" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b970f2c8cc794782955cea7e38004a12" is_pre_election: true
I20260807 08:32:25.201829 25130 leader_election.cc:302] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:32:25.202069 25724 raft_consensus.cc:2726] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 08:32:25.202365 25006 catalog_manager.cc:5495] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to b970f2c8cc794782955cea7e38004a12 (127.1.157.66). New cstate: current_term: 1 leader_uuid: "b970f2c8cc794782955cea7e38004a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } health_report { overall_health: UNKNOWN } } }
I20260807 08:32:25.217482 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.214724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.1.157.66:37005)
0807 08:32:25.214838 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:25.214840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:25.214852 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:32:25.214976 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:25.214978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:25.215021 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:25.215096 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:25.215268 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.215707 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.216432 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:25.216626 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:25.216686 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:25.216886 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:25.217209 (+ 323us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:32:25.217286 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:25.217318 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 08:32:25.217356 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:32:25.222054 25629 raft_consensus.cc:1261] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Refusing update from remote peer b970f2c8cc794782955cea7e38004a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:25.222088 25226 raft_consensus.cc:1261] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Refusing update from remote peer b970f2c8cc794782955cea7e38004a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:25.222625 25733 consensus_queue.cc:1035] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:25.222775 25723 consensus_queue.cc:1035] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:25.231588 25740 mvcc.cc:205] Tried to move back new op lower bound from 7315830969224347648 to 7315830969138937856. Current Snapshot: MvccSnapshot[applied={T|T < 7315830969224347648}]
I20260807 08:32:25.232236 25741 mvcc.cc:205] Tried to move back new op lower bound from 7315830969224347648 to 7315830969138937856. Current Snapshot: MvccSnapshot[applied={T|T < 7315830969224347648}]
I20260807 08:32:25.236902 25006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:25.237051 25006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:25.239229 25006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Sending DeleteTablet for 3 replicas of tablet e398cbf86e7944018209e7d5a83c40ba
I20260807 08:32:25.240000 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 25111
I20260807 08:32:25.239992 25197 tablet_service.cc:1514] Processing DeleteTablet for tablet e398cbf86e7944018209e7d5a83c40ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:25 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60826
I20260807 08:32:25.240000 25599 tablet_service.cc:1514] Processing DeleteTablet for tablet e398cbf86e7944018209e7d5a83c40ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:25 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:52174
I20260807 08:32:25.240415 25397 tablet_service.cc:1514] Processing DeleteTablet for tablet e398cbf86e7944018209e7d5a83c40ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:25 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:44468
I20260807 08:32:25.243902 25752 tablet_replica.cc:330] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: stopping tablet replica
I20260807 08:32:25.244151 25752 raft_consensus.cc:2229] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:25.244406 25752 raft_consensus.cc:2258] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:25.252177 25752 ts_tablet_manager.cc:1905] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:25.254743 25752 ts_tablet_manager.cc:1918] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:25.254845 25752 log.cc:1196] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/e398cbf86e7944018209e7d5a83c40ba
I20260807 08:32:25.255167 25752 ts_tablet_manager.cc:1939] T e398cbf86e7944018209e7d5a83c40ba P f8e9d323e00d484f867ad17d4ff94bea: Deleting consensus metadata
I20260807 08:32:25.256238 24987 catalog_manager.cc:4841] TS f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): tablet e398cbf86e7944018209e7d5a83c40ba (table test-table [id=2ebff11a358b463b8be3097df7002ad2]) successfully deleted
I20260807 08:32:25.258625 25753 tablet_replica.cc:330] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: stopping tablet replica
I20260807 08:32:25.258903 25753 raft_consensus.cc:2229] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 LEADER]: Raft consensus shutting down.
W20260807 08:32:25.258930 24982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:37649 (error 108)
I20260807 08:32:25.259166 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 25302
I20260807 08:32:25.259297 25753 raft_consensus.cc:2258] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:25.264611 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.264348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:37649 (local address 127.0.0.1:60842)
0807 08:32:25.264484 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.264547 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:37649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 08:32:25.264755 24982 catalog_manager.cc:4584] TS 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e398cbf86e7944018209e7d5a83c40ba: Network error: Client connection negotiation failed: client connection to 127.1.157.65:37649: connect: Connection refused (error 111)
I20260807 08:32:25.268180 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 25500
I20260807 08:32:25.276823 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:25.268291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.0.0.1:44484)
0807 08:32:25.276696 (+ 8405us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:25.276752 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.66:37005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8327}
I20260807 08:32:25.281603 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 24961
I20260807 08:32:45.291363 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--webserver_interface=127.1.157.126
--webserver_port=37553
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:32:45.399600 25756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:45.399824 25756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:45.399863 25756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:45.403731 25756 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:32:45.403775 25756 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:32:45.403789 25756 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:32:45.403801 25756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:32:45.403818 25756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:45.403844 25756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:45.403874 25756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:32:45.403899 25756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:32:45.408812 25756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=37553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:45.409663 25756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:45.410576 25756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:32:45.423524 25756 server_base.cc:1034] running on GCE node
W20260807 08:32:45.423472 25772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.423499 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.423472 25769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:45.424322 25756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:45.425365 25756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:45.426523 25756 hybrid_clock.cc:648] HybridClock initialized: now 1786091565426497 us; error 40 us; skew 500 ppm
Aug 07 08:32:45 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:45.429032 25756 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:32:45.429868 25756 webserver.cc:460] Webserver started at http://127.1.157.126:37553/ using document root <none> and password file <none>
I20260807 08:32:45.430433 25756 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:45.430518 25756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:45.434070 25756 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:45.436036 25786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:45.437141 25756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 08:32:45.437254 25756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "ad51bb7ec25e4ef6b9316baa6144da21"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "083b925400bc4b66b602ddce475d42a1"
server_key_iv: "733c7129382f7267c60c9fab579044f4"
server_key_version: "encryptionkey@0"
I20260807 08:32:45.437804 25756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:45.449684 25756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:45.452414 25756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:45.452579 25756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:45.459266 25756 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43983
I20260807 08:32:45.459518 25870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43983 every 8 connection(s)
I20260807 08:32:45.460264 25756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:32:45.464560 25871 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:32:45.466176 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap starting.
I20260807 08:32:45.468539 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 25756
I20260807 08:32:45.469008 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:37649
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=40729
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:45.483889 25871 log.cc:824] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:45.492098 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:45.492439 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap complete.
I20260807 08:32:45.496404 25871 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:45.496771 25871 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Initialized, Role: FOLLOWER
I20260807 08:32:45.497202 25871 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:45.497320 25871 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:32:45.497385 25871 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:32:45.497476 25871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 1 FOLLOWER]: Advancing to term 2
I20260807 08:32:45.499295 25871 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:45.499596 25871 leader_election.cc:302] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad51bb7ec25e4ef6b9316baa6144da21; no voters:
I20260807 08:32:45.499861 25871 leader_election.cc:288] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 08:32:45.499989 25878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 08:32:45.500324 25878 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 LEADER]: Becoming Leader. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Running, Role: LEADER
I20260807 08:32:45.500680 25878 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:32:45.500968 25871 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:32:45.502041 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:32:45.502158 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:45.502914 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad51bb7ec25e4ef6b9316baa6144da21. Latest consensus state: current_term: 2 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:32:45.503270 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
I20260807 08:32:45.505311 25899 catalog_manager.cc:1259] Loaded cluster ID: c7b4608acb6e4e06baa58526afdab13f
I20260807 08:32:45.505357 25899 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: loading cluster ID for follower catalog manager: success
I20260807 08:32:45.507077 25899 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: acquiring CA information for follower catalog manager: success
I20260807 08:32:45.508193 25899 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:32:45.508308 25891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:32:45.510125 25891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:32:45.510829 25891 catalog_manager.cc:1259] Loaded cluster ID: c7b4608acb6e4e06baa58526afdab13f
I20260807 08:32:45.510914 25891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:32:45.511878 25891 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:32:45.512271 25891 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Loaded TSK: 0
I20260807 08:32:45.523604 25891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Generated new TSK 1
I20260807 08:32:45.523874 25891 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:32:45.595763 25876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:45.595995 25876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:45.596048 25876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:45.600101 25876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:45.600158 25876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:45.600215 25876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:32:45.605618 25876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:37649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=40729
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:45.606647 25876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:45.607600 25876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:45.618100 25909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.618131 25913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.618131 25910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:45.618831 25876 server_base.cc:1034] running on GCE node
I20260807 08:32:45.619285 25876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:45.620044 25876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:45.621254 25876 hybrid_clock.cc:648] HybridClock initialized: now 1786091565621223 us; error 44 us; skew 500 ppm
Aug 07 08:32:45 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:45.624609 25876 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:32:45.625697 25876 webserver.cc:460] Webserver started at http://127.1.157.65:40729/ using document root <none> and password file <none>
I20260807 08:32:45.626413 25876 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:45.626529 25876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:45.631752 25876 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:32:45.634301 25927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:45.635835 25876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:32:45.635962 25876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "0e74b23bf2204ec79a25b4169a552c14"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "8e6264c8ae4c459ba478c55738ce168f"
server_key_iv: "7606633326ff7de22e223fa1e9c19a19"
server_key_version: "encryptionkey@0"
I20260807 08:32:45.636662 25876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:45.659759 25876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:45.663163 25876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:45.663337 25876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:45.663962 25876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:45.665132 25946 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:32:45.668989 25876 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:32:45.669061 25876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 08:32:45.669116 25876 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:32:45.671169 25876 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:32:45.671228 25876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 08:32:45.671496 25946 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap starting.
I20260807 08:32:45.681488 25876 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:37649
I20260807 08:32:45.681757 26091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:37649 every 8 connection(s)
I20260807 08:32:45.682446 25876 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:32:45.686494 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 25876
I20260807 08:32:45.686944 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:37005
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=41039
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:45.692626 25946 log.cc:824] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:32:45 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:45.700498 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.684508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:51429 (local address 127.1.157.126:43983)
0807 08:32:45.685320 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:45.685337 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:45.685371 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:32:45.686486 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:45.686501 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:45.686875 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:45.687211 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:45.688885 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.689811 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.691227 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.691938 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.692098 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.694954 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:45.694960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:45.694975 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:45.695007 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:45.697013 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:45.697659 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:45.697665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:45.697670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:45.697816 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:45.698221 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:45.698224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:45.698226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:45.698703 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:45.698837 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:45.699440 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 08:32:45.699722 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":106,"threads_started":1}
I20260807 08:32:45.700508 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.684392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:51429)
0807 08:32:45.684959 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:45.685020 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:45.686007 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:45.687653 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:45.687662 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:45.688096 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:45.688678 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.688699 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.689975 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.689977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.691082 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.691095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.692312 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.692316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.692494 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.693120 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:45.693136 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:45.694740 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:45.697155 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:45.697161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:45.697175 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:45.697527 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:45.697927 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:45.697930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:45.697932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:45.698110 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:45.698832 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:45.698838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:45.699109 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 08:32:45.699306 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 08:32:45.699609 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260807 08:32:45.700874 25946 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:45.701334 25946 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap complete.
I20260807 08:32:45.702551 25946 ts_tablet_manager.cc:1397] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.017s
I20260807 08:32:45.703475 26093 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:45.703698 26093 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:45.704182 26093 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:45.705097 25946 raft_consensus.cc:348] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:45.705530 25946 raft_consensus.cc:729] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e74b23bf2204ec79a25b4169a552c14, State: Initialized, Role: FOLLOWER
I20260807 08:32:45.706018 25946 consensus_queue.cc:260] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:45.706929 25809 ts_manager.cc:194] Registered new tserver with Master: 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649)
I20260807 08:32:45.707063 25946 ts_tablet_manager.cc:1428] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260807 08:32:45.710019 25809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:51429
I20260807 08:32:45.712517 26093 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:45.717167 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.709522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:37649 (local address 127.0.0.1:34046)
0807 08:32:45.712874 (+ 3352us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:45.712921 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:45.713052 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:45.713545 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:45.713551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:45.713572 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:45.713705 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.713712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.714980 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.714982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.715821 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.715831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.716788 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.716791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.716873 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.716901 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 08:32:45.716977 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 08:32:45.717061 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3196,"thread_start_us":99,"threads_started":1}
I20260807 08:32:45.717207 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.709621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.1.157.65:37649)
0807 08:32:45.710549 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:45.710560 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:45.712994 (+ 2434us) server_negotiation.cc:407] Connection header received
0807 08:32:45.713190 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:45.713196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:45.713379 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:45.713520 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:45.713821 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.714872 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.715950 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.716683 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.716790 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.716911 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:45.717005 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:32:45.717086 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":91,"threads_started":1}
I20260807 08:32:45.718253 25998 tablet_service.cc:1514] Processing DeleteTablet for tablet e398cbf86e7944018209e7d5a83c40ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:25 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:34046
I20260807 08:32:45.719079 26108 tablet_replica.cc:330] stopping tablet replica
I20260807 08:32:45.719348 26108 raft_consensus.cc:2229] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:45.719559 26108 raft_consensus.cc:2258] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:45.720398 26108 ts_tablet_manager.cc:1905] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:45.723788 26108 ts_tablet_manager.cc:1918] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:45.723865 26108 log.cc:1196] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal/wals/e398cbf86e7944018209e7d5a83c40ba
I20260807 08:32:45.724083 26108 ts_tablet_manager.cc:1939] T e398cbf86e7944018209e7d5a83c40ba P 0e74b23bf2204ec79a25b4169a552c14: Deleting consensus metadata
I20260807 08:32:45.725556 25787 catalog_manager.cc:4841] TS 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649): tablet e398cbf86e7944018209e7d5a83c40ba (table test-table [id=2ebff11a358b463b8be3097df7002ad2]) successfully deleted
W20260807 08:32:45.803874 26099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:45.804106 26099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:45.804162 26099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:45.808019 26099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:45.808068 26099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:45.808143 26099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:32:45.813108 26099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:37005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=41039
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:45.814121 26099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:45.815062 26099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:45.824182 26117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.824235 26118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:45.824235 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:45.824586 26099 server_base.cc:1034] running on GCE node
I20260807 08:32:45.824965 26099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:45.825577 26099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:45.826735 26099 hybrid_clock.cc:648] HybridClock initialized: now 1786091565826717 us; error 34 us; skew 500 ppm
Aug 07 08:32:45 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:45.829246 26099 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:32:45.830031 26099 webserver.cc:460] Webserver started at http://127.1.157.66:41039/ using document root <none> and password file <none>
I20260807 08:32:45.830608 26099 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:45.830701 26099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:45.834508 26099 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 08:32:45.836480 26133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:45.837595 26099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:45.837697 26099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "b970f2c8cc794782955cea7e38004a12"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "b75585c93c348c5c4a2630cd88e0bbd1"
server_key_iv: "0299a5c4fefcaa57c9a0550ef24b8242"
server_key_version: "encryptionkey@0"
I20260807 08:32:45.838207 26099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:45.852391 26099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:45.854928 26099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:45.855064 26099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:45.855549 26099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:45.856484 26141 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:32:45.860275 26099 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:32:45.860343 26099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260807 08:32:45.860406 26099 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:32:45.861941 26099 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:32:45.861984 26099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 08:32:45.862267 26141 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Bootstrap starting.
I20260807 08:32:45.871953 26099 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37005
I20260807 08:32:45.872018 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37005 every 8 connection(s)
I20260807 08:32:45.872849 26099 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:32:45.874504 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 26099
I20260807 08:32:45.874943 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:42049
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=41551
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:32:45.880046 26141 log.cc:824] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:32:45 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091565, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:45.887230 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.874744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:36049 (local address 127.1.157.126:43983)
0807 08:32:45.874897 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:45.874901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:45.875686 (+ 785us) server_negotiation.cc:407] Connection header received
0807 08:32:45.876840 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:45.876844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:45.876905 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:45.876953 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:45.878199 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.878784 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.879903 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.880150 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.880259 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.883163 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:45.883167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:45.883170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:45.883204 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:45.884934 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:45.885524 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:45.885528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:45.885530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:45.885622 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:45.886007 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:45.886010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:45.886012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:45.886241 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:45.886380 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:45.886920 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 08:32:45.887059 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 08:32:45.887974 26141 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:32:45.888032 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.874891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:36049)
0807 08:32:45.875524 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:45.875581 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:45.876570 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:45.877251 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:45.877257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:45.877585 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:45.878044 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.878057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.878928 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.878932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.879777 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.879786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.880288 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.880292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.880492 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.881154 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:45.881174 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:45.882967 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:45.885062 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:45.885068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:45.885080 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:45.885416 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:45.885729 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:45.885732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:45.885734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:45.885901 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:45.886380 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:45.886386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:45.886656 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 08:32:45.886846 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 08:32:45.887144 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":137,"threads_started":1}
I20260807 08:32:45.888376 26141 tablet_bootstrap.cc:492] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Bootstrap complete.
I20260807 08:32:45.889207 26313 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:45.889446 26313 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:45.889602 26141 ts_tablet_manager.cc:1397] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.004s
I20260807 08:32:45.889959 26313 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:45.891776 25809 ts_manager.cc:194] Registered new tserver with Master: b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
I20260807 08:32:45.891855 26141 raft_consensus.cc:348] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:45.892308 26141 raft_consensus.cc:729] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b970f2c8cc794782955cea7e38004a12, State: Initialized, Role: FOLLOWER
I20260807 08:32:45.892764 26141 consensus_queue.cc:260] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:45.893013 25809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:36049
I20260807 08:32:45.893767 26141 ts_tablet_manager.cc:1428] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:32:45.895545 26313 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:45.898444 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.892681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.0.0.1:43036)
0807 08:32:45.892947 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:45.892962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:45.893039 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:45.894929 (+ 1890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:45.894933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:45.894953 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:45.895198 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.895205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.896180 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.896183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.896956 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:45.896967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.898115 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.898117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:45.898192 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.898209 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 08:32:45.898249 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 08:32:45.898307 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260807 08:32:45.898515 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:45.893347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.1.157.66:37005)
0807 08:32:45.894566 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:45.894574 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:45.894593 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:32:45.894644 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:45.894648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:45.894785 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:45.894923 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:45.895304 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.896079 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.897230 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:45.898001 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:45.898106 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:45.898238 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:45.898285 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 08:32:45.898397 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1138,"thread_start_us":87,"threads_started":1}
I20260807 08:32:45.899449 26211 tablet_service.cc:1514] Processing DeleteTablet for tablet e398cbf86e7944018209e7d5a83c40ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:25 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:43036
I20260807 08:32:45.900187 26327 tablet_replica.cc:330] stopping tablet replica
I20260807 08:32:45.900476 26327 raft_consensus.cc:2229] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:45.900707 26327 raft_consensus.cc:2258] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:45.901726 26327 ts_tablet_manager.cc:1905] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:45.906152 26327 ts_tablet_manager.cc:1918] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:45.906258 26327 log.cc:1196] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/e398cbf86e7944018209e7d5a83c40ba
I20260807 08:32:45.906554 26327 ts_tablet_manager.cc:1939] T e398cbf86e7944018209e7d5a83c40ba P b970f2c8cc794782955cea7e38004a12: Deleting consensus metadata
I20260807 08:32:45.908213 25788 catalog_manager.cc:4841] TS b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): tablet e398cbf86e7944018209e7d5a83c40ba (table test-table [id=2ebff11a358b463b8be3097df7002ad2]) successfully deleted
W20260807 08:32:45.994383 26317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:32:45.994621 26317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:32:45.994678 26317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:32:45.998610 26317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:32:45.998659 26317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:32:45.998735 26317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:32:46.003861 26317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:42049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=41551
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:32:46.004771 26317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:32:46.005710 26317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:32:46.014590 26336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:32:46.014668 26338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:46.014638 26317 server_base.cc:1034] running on GCE node
W20260807 08:32:46.014835 26334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:32:46.015218 26317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:32:46.015789 26317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:32:46.016952 26317 hybrid_clock.cc:648] HybridClock initialized: now 1786091566016936 us; error 26 us; skew 500 ppm
Aug 07 08:32:46 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:32:46.019534 26317 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:32:46.020388 26317 webserver.cc:460] Webserver started at http://127.1.157.67:41551/ using document root <none> and password file <none>
I20260807 08:32:46.020893 26317 fs_manager.cc:359] Metadata directory not provided
I20260807 08:32:46.020969 26317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:32:46.024674 26317 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:32:46.026541 26351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:46.027647 26317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:32:46.027748 26317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "f8e9d323e00d484f867ad17d4ff94bea"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "14b39b5fe9955bb6b4adecf3eab3f589"
server_key_iv: "58f2ac345b9f8d102c47bafa9e826304"
server_key_version: "encryptionkey@0"
I20260807 08:32:46.028242 26317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:32:46.047657 26317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:32:46.050484 26317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:32:46.050643 26317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:32:46.051157 26317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:32:46.051896 26317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:32:46.051941 26317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:46.051990 26317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:32:46.052019 26317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:32:46.060151 26317 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:42049
I20260807 08:32:46.060230 26518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:42049 every 8 connection(s)
I20260807 08:32:46.061039 26317 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:32:46.064508 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 26317
Aug 07 08:32:46 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:32:46.072908 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.065103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.1.157.126:43983)
0807 08:32:46.065353 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.065357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.065398 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:32:46.065527 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.065530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.065571 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.065658 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:46.066409 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.066919 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.067856 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.068441 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.068553 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.069809 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:46.069813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:46.069816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:46.069838 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:46.071347 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.071737 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.071740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.071742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.071836 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.072089 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.072092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.072094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.072260 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:46.072452 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.072629 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.072741 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"spinlock_wait_cycles":9984,"thread_start_us":84,"threads_started":1}
I20260807 08:32:46.073719 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.062635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:52627 (local address 127.1.157.126:43983)
0807 08:32:46.062750 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.062753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.063485 (+ 732us) server_negotiation.cc:407] Connection header received
0807 08:32:46.064699 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.064702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.064760 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.064841 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:46.066168 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.066738 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.067836 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.068048 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.068133 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.070530 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:46.070532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:46.070534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:46.070565 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:46.071847 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.072317 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.072320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.072321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.072390 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.072708 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.072711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.072713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.072893 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:46.072977 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.073475 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.073578 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":1408}
I20260807 08:32:46.074472 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.062932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:52627)
0807 08:32:46.063336 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:46.063374 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:46.064484 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:46.065022 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:46.065030 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:46.065479 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:32:46.065996 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.066009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.066868 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.066870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.067702 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.067711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.068192 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.068196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.068354 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.068846 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:32:46.068863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:32:46.070372 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:32:46.071964 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:46.071970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:46.071982 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:46.072218 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:46.072477 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:32:46.072480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:32:46.072482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:32:46.072608 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:32:46.072979 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:32:46.072985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:32:46.073218 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 08:32:46.073408 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 08:32:46.073675 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260807 08:32:46.075577 26519 heartbeater.cc:347] Connected to a master server at 127.1.157.126:43983
I20260807 08:32:46.075817 26519 heartbeater.cc:464] Registering TS with master...
I20260807 08:32:46.076264 26519 heartbeater.cc:511] Master 127.1.157.126:43983 requested a full tablet report, sending...
I20260807 08:32:46.077215 25809 ts_manager.cc:194] Registered new tserver with Master: f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049)
I20260807 08:32:46.077726 25809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:52627
I20260807 08:32:46.086362 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:32:46.089736 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.086929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.1.157.126:43983)
0807 08:32:46.087032 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.087035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.087201 (+ 166us) server_negotiation.cc:407] Connection header received
0807 08:32:46.087301 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.087303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.087388 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.087470 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:46.087653 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.088117 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.088889 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.089084 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.089147 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.089363 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:46.089473 (+ 110us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 08:32:46.089647 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53432: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 08:32:46.089787 26098 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53432: token expired
W20260807 08:32:46.090091 25692 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.157.126:43983)
I20260807 08:32:46.096112 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.090410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.1.157.126:43983)
0807 08:32:46.090506 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.090509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.090539 (+ 30us) server_negotiation.cc:407] Connection header received
0807 08:32:46.090619 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.090621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.090652 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.090712 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:32:46.091228 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.091665 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.092329 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.092515 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.092580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.093683 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:32:46.093685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:32:46.093686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:32:46.093700 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:32:46.094828 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.095169 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.095171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.095172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.095247 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:32:46.095483 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:32:46.095485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:32:46.095486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:32:46.095662 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:32:46.095760 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.095932 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.096013 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:32:46.098485 25809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:32:46.101625 25809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:32:46.109016 26211 tablet_service.cc:1467] Processing CreateTablet for tablet 57f8a9cf62b9465da13bf393f9ece1c8 (DEFAULT_TABLE table=test-table [id=7084ccfaaf8f4d0091c6831897137e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:46.109326 25998 tablet_service.cc:1467] Processing CreateTablet for tablet 57f8a9cf62b9465da13bf393f9ece1c8 (DEFAULT_TABLE table=test-table [id=7084ccfaaf8f4d0091c6831897137e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:46.109642 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f8a9cf62b9465da13bf393f9ece1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:46.109833 25998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f8a9cf62b9465da13bf393f9ece1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:32:46.112669 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.108249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.0.0.1:42256)
0807 08:32:46.108466 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:46.108480 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:46.108549 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:46.109285 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:46.109288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:46.109311 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:46.109385 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.109390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.110764 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.110768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.111475 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.111484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.112289 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.112292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.112379 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.112401 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:32:46.112469 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 08:32:46.112543 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 08:32:46.112731 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.108521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.1.157.67:42049)
0807 08:32:46.108852 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.108864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.108885 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:32:46.108958 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.108964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.109124 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.109521 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:46.109679 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.110641 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.111602 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.112158 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.112258 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.112399 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.112508 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.112601 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":103,"threads_started":1}
I20260807 08:32:46.112888 25946 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap starting.
I20260807 08:32:46.113577 25946 tablet_bootstrap.cc:654] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:46.114303 26141 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Bootstrap starting.
I20260807 08:32:46.114832 25946 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: No bootstrap required, opened a new log
I20260807 08:32:46.114892 25946 ts_tablet_manager.cc:1397] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 08:32:46.114977 26141 tablet_bootstrap.cc:654] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:46.115301 25946 raft_consensus.cc:348] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.115397 25946 raft_consensus.cc:374] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:46.115432 25946 raft_consensus.cc:729] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e74b23bf2204ec79a25b4169a552c14, State: Initialized, Role: FOLLOWER
I20260807 08:32:46.115535 25946 consensus_queue.cc:260] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.115785 25946 ts_tablet_manager.cc:1428] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 08:32:46.117117 26141 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: No bootstrap required, opened a new log
I20260807 08:32:46.117218 26141 ts_tablet_manager.cc:1397] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 08:32:46.117601 26141 raft_consensus.cc:348] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.117717 26141 raft_consensus.cc:374] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:46.117764 26141 raft_consensus.cc:729] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b970f2c8cc794782955cea7e38004a12, State: Initialized, Role: FOLLOWER
I20260807 08:32:46.117890 26141 consensus_queue.cc:260] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.118161 26141 ts_tablet_manager.cc:1428] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 08:32:46.118739 26423 tablet_service.cc:1467] Processing CreateTablet for tablet 57f8a9cf62b9465da13bf393f9ece1c8 (DEFAULT_TABLE table=test-table [id=7084ccfaaf8f4d0091c6831897137e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:32:46.119714 26423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f8a9cf62b9465da13bf393f9ece1c8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 08:32:46.123760 26314 tablet.cc:2369] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:46.126024 26534 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Bootstrap starting.
I20260807 08:32:46.127779 26534 tablet_bootstrap.cc:654] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Neither blocks nor log segments found. Creating new log.
I20260807 08:32:46.128806 26534 log.cc:824] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Log is configured to *not* fsync() on all Append() calls
I20260807 08:32:46.130384 26534 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: No bootstrap required, opened a new log
I20260807 08:32:46.130558 26534 ts_tablet_manager.cc:1397] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:32:46.133246 26534 raft_consensus.cc:348] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.133451 26534 raft_consensus.cc:374] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:32:46.133512 26534 raft_consensus.cc:729] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e9d323e00d484f867ad17d4ff94bea, State: Initialized, Role: FOLLOWER
I20260807 08:32:46.133919 26534 consensus_queue.cc:260] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.134493 26519 heartbeater.cc:503] Master 127.1.157.126:43983 was elected leader, sending a full tablet report...
I20260807 08:32:46.134773 26534 ts_tablet_manager.cc:1428] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:32:46.150048 26103 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:32:46.150198 26103 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.151124 26103 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
I20260807 08:32:46.154596 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.151305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.1.157.65:42079)
0807 08:32:46.151442 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:46.151457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:46.151540 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:46.151812 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:46.151815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:46.151835 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:46.151946 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.151952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.152720 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.152723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.153520 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.153530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.154194 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.154198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.154313 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.154355 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 08:32:46.154418 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:32:46.154475 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 08:32:46.154636 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.151407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:42079 (local address 127.1.157.66:37005)
0807 08:32:46.151524 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.151528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.151542 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:32:46.151658 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.151662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.151707 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.151801 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:46.152053 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.152609 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.153652 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.154070 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.154159 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.154199 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.154443 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.154524 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 08:32:46.154701 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.151418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40487 (local address 127.1.157.67:42049)
0807 08:32:46.151524 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.151528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.151810 (+ 282us) server_negotiation.cc:407] Connection header received
0807 08:32:46.151889 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.151892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.151933 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.152007 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:32:46.152258 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.152755 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.153752 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.154147 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.154252 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.154299 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.154541 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.154607 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:32:46.154774 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.151310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.1.157.65:40487)
0807 08:32:46.151704 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:46.151717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:32:46.151817 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:32:46.152025 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:32:46.152028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:32:46.152044 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:32:46.152162 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.152168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.152868 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.152871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.153635 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:32:46.153645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.154298 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.154301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:32:46.154411 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.154438 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:32:46.154509 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:32:46.154568 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":162,"thread_start_us":61,"threads_started":1}
I20260807 08:32:46.155121 26245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f8a9cf62b9465da13bf393f9ece1c8" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b970f2c8cc794782955cea7e38004a12" is_pre_election: true
I20260807 08:32:46.155295 26450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f8a9cf62b9465da13bf393f9ece1c8" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e9d323e00d484f867ad17d4ff94bea" is_pre_election: true
I20260807 08:32:46.155392 26245 raft_consensus.cc:2454] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e74b23bf2204ec79a25b4169a552c14 in term 0.
I20260807 08:32:46.155524 26450 raft_consensus.cc:2454] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e74b23bf2204ec79a25b4169a552c14 in term 0.
I20260807 08:32:46.155803 25931 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14, b970f2c8cc794782955cea7e38004a12; no voters:
I20260807 08:32:46.156066 26103 raft_consensus.cc:2781] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:32:46.156141 26103 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:32:46.156184 26103 raft_consensus.cc:3037] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:46.157107 26103 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.157400 26103 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 election: Requested vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
I20260807 08:32:46.157691 26245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f8a9cf62b9465da13bf393f9ece1c8" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b970f2c8cc794782955cea7e38004a12"
I20260807 08:32:46.157699 26450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f8a9cf62b9465da13bf393f9ece1c8" candidate_uuid: "0e74b23bf2204ec79a25b4169a552c14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8e9d323e00d484f867ad17d4ff94bea"
I20260807 08:32:46.157788 26450 raft_consensus.cc:3037] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:46.157804 26245 raft_consensus.cc:3037] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:32:46.158849 26245 raft_consensus.cc:2454] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e74b23bf2204ec79a25b4169a552c14 in term 1.
I20260807 08:32:46.158927 26450 raft_consensus.cc:2454] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e74b23bf2204ec79a25b4169a552c14 in term 1.
I20260807 08:32:46.159089 25931 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14, b970f2c8cc794782955cea7e38004a12; no voters:
I20260807 08:32:46.159248 26103 raft_consensus.cc:2781] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:32:46.159432 26103 raft_consensus.cc:686] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 LEADER]: Becoming Leader. State: Replica: 0e74b23bf2204ec79a25b4169a552c14, State: Running, Role: LEADER
I20260807 08:32:46.159652 26103 consensus_queue.cc:237] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:32:46.162142 25809 catalog_manager.cc:5495] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65). New cstate: current_term: 1 leader_uuid: "0e74b23bf2204ec79a25b4169a552c14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } health_report { overall_health: HEALTHY } } }
I20260807 08:32:46.179064 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.175448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.1.157.65:37649)
0807 08:32:46.175623 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 08:32:46.175626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:32:46.175640 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:32:46.175865 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:32:46.175867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:32:46.175920 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:32:46.176032 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:32:46.176559 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.177036 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.177878 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:32:46.178080 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:32:46.178144 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:32:46.178331 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:32:46.178748 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:32:46.178837 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:32:46.178895 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 08:32:46.178944 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 08:32:46.182554 26245 raft_consensus.cc:1261] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Refusing update from remote peer 0e74b23bf2204ec79a25b4169a552c14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:46.182559 26450 raft_consensus.cc:1261] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Refusing update from remote peer 0e74b23bf2204ec79a25b4169a552c14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:32:46.183169 26103 consensus_queue.cc:1035] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:32:46.183348 26540 consensus_queue.cc:1035] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 08:32:46.183698 26095 tablet.cc:2369] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:32:46.189106 26547 mvcc.cc:205] Tried to move back new op lower bound from 7315831055078240256 to 7315831054992166912. Current Snapshot: MvccSnapshot[applied={T|T < 7315831055078240256}]
I20260807 08:32:46.191790 26548 mvcc.cc:205] Tried to move back new op lower bound from 7315831055078240256 to 7315831054992166912. Current Snapshot: MvccSnapshot[applied={T|T < 7315831055078240256}]
I20260807 08:32:46.196339 25806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:46.196498 25806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:32:46.198791 25806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Sending DeleteTablet for 3 replicas of tablet 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:32:46.199350 26211 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f8a9cf62b9465da13bf393f9ece1c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:46 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:43036
I20260807 08:32:46.199398 26423 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f8a9cf62b9465da13bf393f9ece1c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:32:46 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:42256
I20260807 08:32:46.199623 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 25876
I20260807 08:32:46.199775 26327 tablet_replica.cc:330] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: stopping tablet replica
I20260807 08:32:46.199833 26561 tablet_replica.cc:330] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: stopping tablet replica
I20260807 08:32:46.199877 26327 raft_consensus.cc:2229] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:46.199980 26327 raft_consensus.cc:2258] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:46.200073 26561 raft_consensus.cc:2229] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:32:46.200296 26561 raft_consensus.cc:2258] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:32:46.206187 26327 ts_tablet_manager.cc:1905] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:46.209196 26327 ts_tablet_manager.cc:1918] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:46.209298 26327 log.cc:1196] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:32:46.209605 26327 ts_tablet_manager.cc:1939] T 57f8a9cf62b9465da13bf393f9ece1c8 P b970f2c8cc794782955cea7e38004a12: Deleting consensus metadata
I20260807 08:32:46.210371 25788 catalog_manager.cc:4841] TS b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): tablet 57f8a9cf62b9465da13bf393f9ece1c8 (table test-table [id=7084ccfaaf8f4d0091c6831897137e2d]) successfully deleted
I20260807 08:32:46.213193 26561 ts_tablet_manager.cc:1905] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:32:46.215217 26561 ts_tablet_manager.cc:1918] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:32:46.215301 26561 log.cc:1196] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:32:46.215553 26561 ts_tablet_manager.cc:1939] T 57f8a9cf62b9465da13bf393f9ece1c8 P f8e9d323e00d484f867ad17d4ff94bea: Deleting consensus metadata
I20260807 08:32:46.216568 25791 catalog_manager.cc:4841] TS f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): tablet 57f8a9cf62b9465da13bf393f9ece1c8 (table test-table [id=7084ccfaaf8f4d0091c6831897137e2d]) successfully deleted
W20260807 08:32:46.218367 25787 connection.cc:535] client connection to 127.1.157.65:37649 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 08:32:46.218513 25787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:32:46.219219 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 26099
I20260807 08:32:46.219461 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:32:46.219187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:37649 (local address 127.0.0.1:60808)
0807 08:32:46.219323 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 08:32:46.219389 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:37649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 08:32:46.221966 25787 catalog_manager.cc:4584] TS 0e74b23bf2204ec79a25b4169a552c14 (127.1.157.65:37649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57f8a9cf62b9465da13bf393f9ece1c8: Network error: Client connection negotiation failed: client connection to 127.1.157.65:37649: connect: Connection refused (error 111)
I20260807 08:32:46.227207 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 26317
I20260807 08:32:46.237443 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 25756
I20260807 08:33:06.249042 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--webserver_interface=127.1.157.126
--webserver_port=37553
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:06.356549 26563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:06.356770 26563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:06.356806 26563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:06.360674 26563 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 08:33:06.360716 26563 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 08:33:06.360730 26563 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 08:33:06.360744 26563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:06.360775 26563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:06.360805 26563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:06.360832 26563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:06.360862 26563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:06.365785 26563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=37553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:06.366693 26563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:06.367566 26563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:06.375267 26575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:06.375248 26572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.375310 26563 server_base.cc:1034] running on GCE node
W20260807 08:33:06.375233 26570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.376032 26563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:06.376997 26563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:06.378150 26563 hybrid_clock.cc:648] HybridClock initialized: now 1786091586378126 us; error 38 us; skew 500 ppm
Aug 07 08:33:06 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:06.380628 26563 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:06.381410 26563 webserver.cc:460] Webserver started at http://127.1.157.126:37553/ using document root <none> and password file <none>
I20260807 08:33:06.381865 26563 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:06.381920 26563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:06.385278 26563 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:06.387111 26585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.388164 26563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:06.388278 26563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "ad51bb7ec25e4ef6b9316baa6144da21"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "083b925400bc4b66b602ddce475d42a1"
server_key_iv: "733c7129382f7267c60c9fab579044f4"
server_key_version: "encryptionkey@0"
I20260807 08:33:06.388727 26563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:06.406386 26563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:06.408654 26563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:06.408783 26563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:06.415033 26563 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43983
I20260807 08:33:06.415287 26665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43983 every 8 connection(s)
I20260807 08:33:06.415951 26563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:06.419763 26666 sys_catalog.cc:263] Verifying existing consensus state
I20260807 08:33:06.421238 26666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap starting.
I20260807 08:33:06.425657 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 26563
I20260807 08:33:06.426129 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:37649
--local_ip_for_outbound_sockets=127.1.157.65
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=40729
--webserver_interface=127.1.157.65
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:33:06.439396 26666 log.cc:824] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:06.450960 26666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:33:06.451301 26666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Bootstrap complete.
I20260807 08:33:06.455101 26666 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:33:06.455475 26666 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Initialized, Role: FOLLOWER
I20260807 08:33:06.455924 26666 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:33:06.456049 26666 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:06.456122 26666 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:06.456211 26666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 2 FOLLOWER]: Advancing to term 3
I20260807 08:33:06.457975 26666 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:33:06.458295 26666 leader_election.cc:302] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad51bb7ec25e4ef6b9316baa6144da21; no voters:
I20260807 08:33:06.458618 26666 leader_election.cc:288] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 08:33:06.458742 26671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 08:33:06.458972 26671 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [term 3 LEADER]: Becoming Leader. State: Replica: ad51bb7ec25e4ef6b9316baa6144da21, State: Running, Role: LEADER
I20260807 08:33:06.459316 26671 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } }
I20260807 08:33:06.459725 26666 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:06.460638 26673 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:33:06.460763 26673 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:06.461120 26674 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad51bb7ec25e4ef6b9316baa6144da21. Latest consensus state: current_term: 3 leader_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad51bb7ec25e4ef6b9316baa6144da21" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43983 } } }
I20260807 08:33:06.461258 26674 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:06.461930 26683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:06.466609 26683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:06.467244 26683 catalog_manager.cc:1259] Loaded cluster ID: c7b4608acb6e4e06baa58526afdab13f
I20260807 08:33:06.467305 26683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:06.469748 26683 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:06.470484 26683 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Loaded TSK: 0
I20260807 08:33:06.470566 26683 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad51bb7ec25e4ef6b9316baa6144da21: Loaded TSK: 1
F20260807 08:33:06.471365 26683 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6f56fc69ad google::LogMessage::Fail() at ??:0
@ 0x7f6f56fc988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6f56fc64de google::LogMessage::Flush() at ??:0
@ 0x7f6f56fc7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6f59c8a18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6f59c82f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6f6192b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6f6190599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6f61906178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6f61947df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6f6196682b std::function<>::operator()() at ??:0
@ 0x7f6f61905b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6f61906f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6f61903579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6f61947640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f592183d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6f59218c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6f5921a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f59209512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6f5a84d6db start_thread at ??:0
@ 0x7f6f55ac461f clone at ??:0
W20260807 08:33:06.557507 26668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:06.557791 26668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:06.557855 26668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:06.563709 26668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:06.563776 26668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:06.563869 26668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:06.571223 26668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:37649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=40729
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:06.572431 26668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:06.573697 26668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:06.584193 26732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.584235 26668 server_base.cc:1034] running on GCE node
W20260807 08:33:06.584306 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:06.584445 26735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.584832 26668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:06.585556 26668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:06.586743 26668 hybrid_clock.cc:648] HybridClock initialized: now 1786091586586712 us; error 44 us; skew 500 ppm
Aug 07 08:33:06 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:06.589802 26668 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:33:06.590612 26668 webserver.cc:460] Webserver started at http://127.1.157.65:40729/ using document root <none> and password file <none>
I20260807 08:33:06.591142 26668 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:06.591205 26668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:06.594945 26668 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:33:06.596984 26744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.598248 26668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:06.598383 26668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "0e74b23bf2204ec79a25b4169a552c14"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "8e6264c8ae4c459ba478c55738ce168f"
server_key_iv: "7606633326ff7de22e223fa1e9c19a19"
server_key_version: "encryptionkey@0"
I20260807 08:33:06.598951 26668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:06.611282 26668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:06.614745 26668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:06.614931 26668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:06.615553 26668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:06.616766 26753 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 08:33:06.620879 26668 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 08:33:06.620940 26668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260807 08:33:06.620997 26668 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 08:33:06.622712 26668 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 08:33:06.622766 26668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:06.626143 26753 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap starting.
I20260807 08:33:06.633863 26668 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:37649
I20260807 08:33:06.633998 26926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:37649 every 8 connection(s)
I20260807 08:33:06.634980 26668 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:06.636050 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 26668
I20260807 08:33:06.636520 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:37005
--local_ip_for_outbound_sockets=127.1.157.66
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=41039
--webserver_interface=127.1.157.66
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:33:06.650171 26753 log.cc:824] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:06.658975 26753 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 08:33:06.659399 26753 tablet_bootstrap.cc:492] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Bootstrap complete.
I20260807 08:33:06.660811 26753 ts_tablet_manager.cc:1397] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Time spent bootstrapping tablet: real 0.035s user 0.028s sys 0.000s
I20260807 08:33:06.663955 26753 raft_consensus.cc:348] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:06.664487 26753 raft_consensus.cc:729] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e74b23bf2204ec79a25b4169a552c14, State: Initialized, Role: FOLLOWER
I20260807 08:33:06.664985 26753 consensus_queue.cc:260] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:06.665975 26753 ts_tablet_manager.cc:1428] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260807 08:33:06.713480 26932 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:33:06.637603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:47815)
0807 08:33:06.638166 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.638225 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:06.639244 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:06.712498 (+ 73254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":141,"threads_started":1}
W20260807 08:33:06.713975 26929 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:43983 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:43983: Client connection negotiation failed: client connection to 127.1.157.126:43983: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 08:33:06.714747 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:06.714464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:50567)
0807 08:33:06.714598 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.714681 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 08:33:06.715418 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:06.715195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:59751)
0807 08:33:06.715306 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.715359 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 08:33:06.715579 26929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 08:33:06.754318 26933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:06.754563 26933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:06.754602 26933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:06.758440 26933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:06.758487 26933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:06.758541 26933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:06.763475 26933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:37005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=41039
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:06.764307 26933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:06.765146 26933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:06.773926 26941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:06.773936 26947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.774077 26933 server_base.cc:1034] running on GCE node
W20260807 08:33:06.774273 26945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.774659 26933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:06.775210 26933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:06.776377 26933 hybrid_clock.cc:648] HybridClock initialized: now 1786091586776358 us; error 32 us; skew 500 ppm
Aug 07 08:33:06 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:06.778827 26933 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:33:06.779619 26933 webserver.cc:460] Webserver started at http://127.1.157.66:41039/ using document root <none> and password file <none>
I20260807 08:33:06.780102 26933 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:06.780167 26933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:06.783617 26933 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:33:06.785414 26958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.786530 26933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:33:06.786640 26933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "b970f2c8cc794782955cea7e38004a12"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "b75585c93c348c5c4a2630cd88e0bbd1"
server_key_iv: "0299a5c4fefcaa57c9a0550ef24b8242"
server_key_version: "encryptionkey@0"
I20260807 08:33:06.787122 26933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:06.800344 26933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:06.802908 26933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:06.803063 26933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:06.803570 26933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:06.804301 26933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:06.804345 26933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.804394 26933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:06.804422 26933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.814246 26933 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:37005
I20260807 08:33:06.814278 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:37005 every 8 connection(s)
I20260807 08:33:06.815102 26933 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:06.818233 27118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:06.816917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:47963)
0807 08:33:06.817511 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.817605 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":103,"threads_started":1}
W20260807 08:33:06.818674 27116 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:43983 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:43983: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
I20260807 08:33:06.819164 27118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:06.818973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:59809)
0807 08:33:06.819066 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.819109 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:33:06.819684 27118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:06.819523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:51549)
0807 08:33:06.819596 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:06.819640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 08:33:06.819832 27116 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:33:06.820436 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 26933
I20260807 08:33:06.820793 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:42049
--local_ip_for_outbound_sockets=127.1.157.67
--tserver_master_addrs=127.1.157.126:43983
--webserver_port=41551
--webserver_interface=127.1.157.67
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:06.926467 27119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:06.926682 27119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:06.926721 27119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:06.930511 27119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:06.930553 27119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:06.930604 27119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:06.935839 27119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:42049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=41551
--tserver_master_addrs=127.1.157.126:43983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:06.936731 27119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:06.937575 27119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:06.946254 27126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:06.946262 27129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:06.946267 27127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:06.946470 27119 server_base.cc:1034] running on GCE node
I20260807 08:33:06.946950 27119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:06.947494 27119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:06.948645 27119 hybrid_clock.cc:648] HybridClock initialized: now 1786091586948632 us; error 24 us; skew 500 ppm
Aug 07 08:33:06 dist-test-slave-6zbq krb5kdc[24948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:06.951115 27119 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:33:06.951905 27119 webserver.cc:460] Webserver started at http://127.1.157.67:41551/ using document root <none> and password file <none>
I20260807 08:33:06.952391 27119 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:06.952456 27119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:06.955989 27119 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:06.957852 27140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.958999 27119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:06.959101 27119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "f8e9d323e00d484f867ad17d4ff94bea"
format_stamp: "Formatted at 2026-08-07 08:32:24 on dist-test-slave-6zbq"
server_key: "14b39b5fe9955bb6b4adecf3eab3f589"
server_key_iv: "58f2ac345b9f8d102c47bafa9e826304"
server_key_version: "encryptionkey@0"
I20260807 08:33:06.959590 27119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:06.985839 27119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:06.988559 27119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:06.988710 27119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:06.989224 27119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:06.990011 27119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:06.990057 27119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.990095 27119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:06.990114 27119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:06.998389 27119 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:42049
I20260807 08:33:06.998445 27318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:42049 every 8 connection(s)
I20260807 08:33:06.999296 27119 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:33:07.002619 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:07.001174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:60953)
0807 08:33:07.001809 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:07.001909 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":103,"threads_started":1}
W20260807 08:33:07.003027 27323 heartbeater.cc:644] Failed to heartbeat to 127.1.157.126:43983 (0 consecutive failures): Network error: Failed to ping master at 127.1.157.126:43983: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
I20260807 08:33:07.003561 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:07.003371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:51665)
0807 08:33:07.003464 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:07.003509 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 08:33:07.004073 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:07.003902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:38973)
0807 08:33:07.003986 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:07.004025 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 08:33:07.004230 27323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 08:33:07.007266 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 27119
I20260807 08:33:07.716473 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:07.716068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:34491)
0807 08:33:07.716319 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:07.716371 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260807 08:33:07.820612 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:07.820270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:51375)
0807 08:33:07.820476 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:07.820524 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
I20260807 08:33:08.004988 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.004651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:51181)
0807 08:33:08.004855 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:08.004902 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260807 08:33:08.138895 27335 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:08.139104 27335 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:08.140273 27335 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
Aug 07 08:33:08 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.66@KRBTEST.COM
Aug 07 08:33:08 dist-test-slave-6zbq krb5kdc[24948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091586, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.67@KRBTEST.COM
I20260807 08:33:08.157162 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.140480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:37005 (local address 127.1.157.65:60641)
0807 08:33:08.140759 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:08.140772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:08.140878 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:08.142363 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:08.142369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:08.142894 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:08.143384 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:08.143398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.144766 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.144768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:08.145704 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:08.145713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.146388 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.146392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:08.146621 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:08.151326 (+ 4705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:08.151349 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:08.152801 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:08.154786 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:08.154791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:08.154804 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:08.155114 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:08.155455 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:08.155459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:08.155636 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:08.156262 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:08.156268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:08.156509 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 08:33:08.156661 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 08:33:08.156962 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":129,"spinlock_wait_cycles":23424,"thread_start_us":60,"threads_started":1}
I20260807 08:33:08.157195 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.140586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:38821 (local address 127.1.157.67:42049)
0807 08:33:08.140897 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:08.140925 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:08.140969 (+ 44us) server_negotiation.cc:407] Connection header received
0807 08:33:08.141666 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:08.141672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:08.141997 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:08.142279 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:08.143537 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.144553 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.145819 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.146258 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.146431 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:08.153345 (+ 6914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:08.153349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:08.153360 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:08.153391 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:08.155108 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155552 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:08.155558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:08.155563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:08.155688 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155946 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:08.155947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:08.155949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:08.156319 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:08.156438 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:08.156800 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 08:33:08.157039 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":59,"threads_started":1}
I20260807 08:33:08.157202 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.140757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:60641 (local address 127.1.157.66:37005)
0807 08:33:08.141098 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:08.141117 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:08.141148 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:33:08.141760 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:08.141765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:08.142039 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:08.142310 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:08.143536 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.144638 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.145818 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.146261 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.146431 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:08.153016 (+ 6585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:08.153022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:08.153036 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:08.153068 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:08.154673 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155217 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:08.155223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:08.155238 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:08.155371 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155722 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:08.155725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:08.155726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:08.156143 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:08.156247 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:08.156784 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 08:33:08.157015 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":120,"threads_started":1}
I20260807 08:33:08.157184 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.140417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:42049 (local address 127.1.157.65:38821)
0807 08:33:08.140521 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:08.140536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:08.140638 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:08.142362 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:08.142369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:08.142893 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:08.143384 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:08.143398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.144680 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.144683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:08.145702 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:08.145712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:08.146386 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:08.146389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:08.146620 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:08.151331 (+ 4711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:08.151345 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:08.153147 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:08.155223 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:08.155228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:08.155454 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:08.155782 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:08.155785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:08.155787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:08.155866 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:08.156418 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:08.156420 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:08.156504 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 08:33:08.156660 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:33:08.157034 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":331,"spinlock_wait_cycles":7168}
W20260807 08:33:08.159660 26747 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
W20260807 08:33:08.159807 26749 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:33:08.159884 26749 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:33:08.160110 27335 raft_consensus.cc:2726] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:33:08.717425 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.717071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:60907)
0807 08:33:08.717291 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:08.717338 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":65,"threads_started":1}
I20260807 08:33:08.821511 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:08.821165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:49479)
0807 08:33:08.821386 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:08.821435 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
I20260807 08:33:09.005937 27342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:09.005572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:57259)
0807 08:33:09.005805 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:09.005854 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260807 08:33:09.718438 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:09.718050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:57789)
0807 08:33:09.718274 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:09.718328 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260807 08:33:09.822520 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:09.822112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:60545)
0807 08:33:09.822368 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:09.822423 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260807 08:33:09.969360 27347 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:09.969484 27347 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:09.969873 27347 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
W20260807 08:33:09.970654 26749 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
W20260807 08:33:09.970772 26747 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:33:09.970908 26747 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:33:09.971040 27347 raft_consensus.cc:2726] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:33:10.006875 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:10.006514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:58847)
0807 08:33:10.006734 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:10.006785 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":66,"threads_started":1}
I20260807 08:33:10.719435 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:10.719056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:49385)
0807 08:33:10.719286 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:10.719340 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260807 08:33:10.823419 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:10.823086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:56471)
0807 08:33:10.823295 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:10.823342 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260807 08:33:11.007882 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:11.007474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:43425)
0807 08:33:11.007746 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:11.007798 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
I20260807 08:33:11.720660 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:11.720190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:36123)
0807 08:33:11.720488 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:11.720539 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":87,"threads_started":1}
I20260807 08:33:11.761251 27356 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:11.761361 27356 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:11.761725 27356 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
W20260807 08:33:11.762519 26749 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
W20260807 08:33:11.762642 26747 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:33:11.762733 26747 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:33:11.762923 27356 raft_consensus.cc:2726] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:33:11.824339 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:11.823994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:59039)
0807 08:33:11.824213 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:11.824262 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260807 08:33:12.008802 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:12.008455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:59113)
0807 08:33:12.008675 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:12.008723 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":66,"threads_started":1}
I20260807 08:33:12.721648 27359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:12.721289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:40245)
0807 08:33:12.721505 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:12.721555 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260807 08:33:12.825345 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:12.824956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:41779)
0807 08:33:12.825203 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:12.825257 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
I20260807 08:33:13.009779 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:13.009408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:58631)
0807 08:33:13.009647 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:13.009698 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260807 08:33:13.722760 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:13.722332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:42943)
0807 08:33:13.722595 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:13.722650 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260807 08:33:13.826406 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:13.825968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:36409)
0807 08:33:13.826246 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:13.826296 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":74,"threads_started":1}
I20260807 08:33:14.010842 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:14.010433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:52333)
0807 08:33:14.010708 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:14.010759 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":65,"threads_started":1}
I20260807 08:33:14.723807 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:14.723401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:57055)
0807 08:33:14.723659 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:14.723713 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260807 08:33:14.827333 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:14.826963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:50985)
0807 08:33:14.827202 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:14.827254 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":66,"threads_started":1}
I20260807 08:33:15.011816 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:15.011431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:57901)
0807 08:33:15.011683 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:15.011734 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":85,"threads_started":1}
I20260807 08:33:15.423278 27371 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:15.423410 27371 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:15.423820 27371 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
W20260807 08:33:15.424491 26747 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
W20260807 08:33:15.424602 26749 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:33:15.424665 26749 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:33:15.424794 27371 raft_consensus.cc:2726] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:33:15.724810 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:15.724441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:56101)
0807 08:33:15.724666 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:15.724718 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260807 08:33:15.828296 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:15.827931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:57277)
0807 08:33:15.828161 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:15.828213 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260807 08:33:16.012815 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:16.012441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:56101)
0807 08:33:16.012681 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:16.012732 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260807 08:33:16.725909 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:16.725537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:43671)
0807 08:33:16.725768 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:16.725820 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
I20260807 08:33:16.830271 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:16.828919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:59675)
0807 08:33:16.830131 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:16.830181 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1144,"thread_start_us":1057,"threads_started":1}
I20260807 08:33:17.013878 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:17.013456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:56781)
0807 08:33:17.013729 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:17.013785 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
I20260807 08:33:17.726974 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:17.726582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:47025)
0807 08:33:17.726828 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:17.726880 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
I20260807 08:33:17.831255 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:17.830918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:35943)
0807 08:33:17.831123 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:17.831176 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260807 08:33:18.014895 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:18.014513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:60385)
0807 08:33:18.014763 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:18.014814 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":93,"threads_started":1}
I20260807 08:33:18.727972 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:18.727591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:59807)
0807 08:33:18.727829 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:18.727879 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260807 08:33:18.832369 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:18.831877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:45115)
0807 08:33:18.832238 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:18.832289 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":217,"threads_started":1}
I20260807 08:33:19.015851 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:19.015475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:55881)
0807 08:33:19.015713 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:19.015764 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260807 08:33:19.729058 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:19.728628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:39355)
0807 08:33:19.728904 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:19.728961 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260807 08:33:19.833366 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:19.833002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:53415)
0807 08:33:19.833235 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:19.833285 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260807 08:33:20.016821 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:20.016459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:43355)
0807 08:33:20.016686 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:20.016737 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260807 08:33:20.730103 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:20.729700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:55109)
0807 08:33:20.729955 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:20.730009 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
I20260807 08:33:20.834326 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:20.833953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:49851)
0807 08:33:20.834189 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:20.834240 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260807 08:33:21.017894 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:21.017494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:34067)
0807 08:33:21.017752 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:21.017806 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
I20260807 08:33:21.126657 27403 raft_consensus.cc:482] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:21.126792 27403 raft_consensus.cc:504] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8e9d323e00d484f867ad17d4ff94bea" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 42049 } } peers { permanent_uuid: "b970f2c8cc794782955cea7e38004a12" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 37005 } } peers { permanent_uuid: "0e74b23bf2204ec79a25b4169a552c14" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 37649 } }
I20260807 08:33:21.127185 27403 leader_election.cc:288] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049), b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005)
W20260807 08:33:21.127890 26747 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b970f2c8cc794782955cea7e38004a12 (127.1.157.66:37005): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
W20260807 08:33:21.128005 26749 leader_election.cc:341] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8e9d323e00d484f867ad17d4ff94bea (127.1.157.67:42049): Not found: Tablet not found: 57f8a9cf62b9465da13bf393f9ece1c8
I20260807 08:33:21.128069 26749 leader_election.cc:302] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e74b23bf2204ec79a25b4169a552c14; no voters: b970f2c8cc794782955cea7e38004a12, f8e9d323e00d484f867ad17d4ff94bea
I20260807 08:33:21.128198 27403 raft_consensus.cc:2726] T 57f8a9cf62b9465da13bf393f9ece1c8 P 0e74b23bf2204ec79a25b4169a552c14 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 08:33:21.731292 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:21.730899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.65:44323)
0807 08:33:21.731127 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:21.731188 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260807 08:33:21.835381 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:21.835032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.66:55773)
0807 08:33:21.835256 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:21.835303 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260807 08:33:22.018131 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 26668
I20260807 08:33:22.022434 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:22.018670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:43983 (local address 127.1.157.67:33797)
0807 08:33:22.022267 (+ 3597us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:22.022321 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.126:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3524,"thread_start_us":63,"threads_started":1}
I20260807 08:33:22.031257 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 26933
I20260807 08:33:22.040290 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 27119
2026-08-07T08:33:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59074 ms)
[----------] 30 tests from SecurityITest (237438 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:22 dist-test-slave-6zbq krb5kdc[27413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:22 dist-test-slave-6zbq krb5kdc[27413](info): set up 2 sockets
Aug 07 08:33:22 dist-test-slave-6zbq krb5kdc[27413](info): commencing operation
krb5kdc: starting...
W20260807 08:33:23.163991 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:23.286794 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45537
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:23.395320 27433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:23.395548 27433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:23.395609 27433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:23.399492 27433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:23.399538 27433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:23.399564 27433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:23.399593 27433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:23.399622 27433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:23.404584 27433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:45537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:23.405434 27433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:23.406363 27433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:23.414148 27444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:23.414223 27446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:23.414227 27441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:23.414407 27433 server_base.cc:1034] running on GCE node
I20260807 08:33:23.414971 27433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:23.415913 27433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:23.417071 27433 hybrid_clock.cc:648] HybridClock initialized: now 1786091603417048 us; error 35 us; skew 500 ppm
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:23.419605 27433 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:23.420348 27433 webserver.cc:460] Webserver started at http://127.1.157.126:36257/ using document root <none> and password file <none>
I20260807 08:33:23.420814 27433 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:23.420866 27433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:23.421021 27433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:23.422941 27433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "0201ed0e4d094e8f999ad2c1f76a93e7"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "758ab912719cfe888b4f57f9af8bce6d"
server_key_iv: "3ab4e64405f9d003558abbfe4672b0a1"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.423416 27433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "0201ed0e4d094e8f999ad2c1f76a93e7"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "758ab912719cfe888b4f57f9af8bce6d"
server_key_iv: "3ab4e64405f9d003558abbfe4672b0a1"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.426756 27433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:23.428802 27453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.429893 27433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:23.429992 27433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "0201ed0e4d094e8f999ad2c1f76a93e7"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "758ab912719cfe888b4f57f9af8bce6d"
server_key_iv: "3ab4e64405f9d003558abbfe4672b0a1"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.430083 27433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:23.446805 27433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:23.449110 27433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:23.449257 27433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:23.455672 27433 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:45537
I20260807 08:33:23.455717 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:45537 every 8 connection(s)
I20260807 08:33:23.456506 27433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:23.459028 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:23.462401 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 27433
I20260807 08:33:23.462510 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:23.462743 1653 external_mini_cluster.cc:1428] Setting key 5fa093385bb6d4a2a1657dd385a1e447
I20260807 08:33:23.464452 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: Bootstrap starting.
I20260807 08:33:23.466516 27537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:23.467186 27537 log.cc:824] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:23.469069 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: No bootstrap required, opened a new log
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:23.473012 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } }
I20260807 08:33:23.473208 27537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:23.473241 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0201ed0e4d094e8f999ad2c1f76a93e7, State: Initialized, Role: FOLLOWER
I20260807 08:33:23.473675 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } }
I20260807 08:33:23.473834 27537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:23.473906 27537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:23.474001 27537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:23.474862 27537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } }
I20260807 08:33:23.475210 27537 leader_election.cc:302] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0201ed0e4d094e8f999ad2c1f76a93e7; no voters:
I20260807 08:33:23.475498 27537 leader_election.cc:288] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:23.475584 27544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:23.475805 27544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [term 1 LEADER]: Becoming Leader. State: Replica: 0201ed0e4d094e8f999ad2c1f76a93e7, State: Running, Role: LEADER
I20260807 08:33:23.476131 27544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } }
I20260807 08:33:23.476317 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:23.463979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.1.157.126:45537)
0807 08:33:23.464371 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:23.464388 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:23.464421 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:23.465054 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:23.465071 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:23.465387 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:23.465666 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:23.466491 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.467502 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.468608 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.469045 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.469186 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:23.471773 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:23.471778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:23.471790 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:23.471815 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:23.473372 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.473850 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.473856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.473861 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.473997 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.474299 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.474301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.474303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.474675 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:23.474823 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:23.475122 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:33:23.475379 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":124,"threads_started":1}
I20260807 08:33:23.476624 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:23.477598 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } } }
I20260807 08:33:23.477712 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:23.477921 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0201ed0e4d094e8f999ad2c1f76a93e7. Latest consensus state: current_term: 1 leader_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0201ed0e4d094e8f999ad2c1f76a93e7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45537 } } }
I20260807 08:33:23.477983 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:23.478809 27557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:23.481652 27557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:23.486330 27557 catalog_manager.cc:1347] Generated new cluster ID: 92e139a62e6c4d40ab735f30fa13ed4a
I20260807 08:33:23.486418 27557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:23.502288 27557 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:23.503103 27557 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:23.514057 27557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: Generated new TSK 0
I20260807 08:33:23.514626 27557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:33:23.546939 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:23.677361 27573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:23.677567 27573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:23.677623 27573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:23.681588 27573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:23.681639 27573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:23.681722 27573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:23.686612 27573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:23.687436 27573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:23.688285 27573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:23.696816 27580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:23.696867 27573 server_base.cc:1034] running on GCE node
W20260807 08:33:23.696902 27581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:23.696801 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:23.697341 27573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:23.697899 27573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:23.699046 27573 hybrid_clock.cc:648] HybridClock initialized: now 1786091603699029 us; error 29 us; skew 500 ppm
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:23.701810 27573 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:33:23.702598 27573 webserver.cc:460] Webserver started at http://127.1.157.65:34509/ using document root <none> and password file <none>
I20260807 08:33:23.703083 27573 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:23.703145 27573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:23.703316 27573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:23.705112 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "f862ca5287754128aeb9e4c823310acf"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "a3ea163d5ab2a16219376fa756074ef8"
server_key_iv: "5080faabf9bee0dc64e5ec0d456520fc"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.705611 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "f862ca5287754128aeb9e4c823310acf"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "a3ea163d5ab2a16219376fa756074ef8"
server_key_iv: "5080faabf9bee0dc64e5ec0d456520fc"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.708948 27573 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:33:23.711090 27591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.712141 27573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:23.712253 27573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "f862ca5287754128aeb9e4c823310acf"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "a3ea163d5ab2a16219376fa756074ef8"
server_key_iv: "5080faabf9bee0dc64e5ec0d456520fc"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.712337 27573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:23.725504 27573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:23.728175 27573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:23.728303 27573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:23.728744 27573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:23.729422 27573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:23.729466 27573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.729501 27573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:23.729521 27573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.737618 27573 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:36669
I20260807 08:33:23.737666 27775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:36669 every 8 connection(s)
I20260807 08:33:23.738438 27573 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:23.742774 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 27573
I20260807 08:33:23.742898 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:23.743209 1653 external_mini_cluster.cc:1428] Setting key 89c03c1770988b48331d458d7c2d64d2
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:23.750864 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:23.739933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:39331 (local address 127.1.157.126:45537)
0807 08:33:23.740053 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:23.740056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:23.741017 (+ 961us) server_negotiation.cc:407] Connection header received
0807 08:33:23.741815 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:23.741818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:23.741869 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:23.741937 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:23.743171 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.743740 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.745061 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.745362 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.745461 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:23.747695 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:23.747698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:23.747700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:23.747722 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:23.748906 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.749406 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.749409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.749411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.749485 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.749796 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.749799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.749801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.750005 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:23.750104 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:23.750577 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 08:33:23.750703 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:33:23.751466 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:23.740263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45537 (local address 127.1.157.65:39331)
0807 08:33:23.740892 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:23.740929 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:23.741636 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:23.742068 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:23.742074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:23.742434 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:23.743022 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:23.743044 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.743862 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.743865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:23.744941 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:23.744952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.745481 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.745483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:23.745607 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:23.746089 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:23.746104 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:23.747547 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:23.749010 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:23.749016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:23.749028 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:23.749314 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:23.749566 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:23.749570 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:23.749571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:23.749709 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:23.750112 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:23.750117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:23.750318 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 08:33:23.750509 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 08:33:23.750741 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":112,"threads_started":1}
I20260807 08:33:23.752547 27776 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45537
I20260807 08:33:23.752779 27776 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:23.753309 27776 heartbeater.cc:511] Master 127.1.157.126:45537 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:33:23.754733 27484 ts_manager.cc:194] Registered new tserver with Master: f862ca5287754128aeb9e4c823310acf (127.1.157.65:36669)
I20260807 08:33:23.755792 27484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:39331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:33:23.768805 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:23.872862 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:23.873095 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:23.873139 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:23.876987 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:23.877032 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:23.877084 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:23.881922 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:23.882725 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:23.883594 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:23.891992 27791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:23.891993 27788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:23.892067 27781 server_base.cc:1034] running on GCE node
W20260807 08:33:23.891993 27789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:23.892546 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:23.893110 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:23.894268 27781 hybrid_clock.cc:648] HybridClock initialized: now 1786091603894245 us; error 37 us; skew 500 ppm
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:23.896668 27781 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:33:23.897439 27781 webserver.cc:460] Webserver started at http://127.1.157.66:34553/ using document root <none> and password file <none>
I20260807 08:33:23.897907 27781 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:23.897974 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:23.898151 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:23.899994 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "e8b52e2f76ab4359b87456a0d1c850ef"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "74867d625457c9fd0c1acecc039dcaec"
server_key_iv: "24deda9b7bf305cf3198e554115d3afd"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.900498 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "e8b52e2f76ab4359b87456a0d1c850ef"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "74867d625457c9fd0c1acecc039dcaec"
server_key_iv: "24deda9b7bf305cf3198e554115d3afd"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.903795 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:33:23.905836 27801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.906885 27781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:23.906982 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "e8b52e2f76ab4359b87456a0d1c850ef"
format_stamp: "Formatted at 2026-08-07 08:33:23 on dist-test-slave-6zbq"
server_key: "74867d625457c9fd0c1acecc039dcaec"
server_key_iv: "24deda9b7bf305cf3198e554115d3afd"
server_key_version: "encryptionkey@0"
I20260807 08:33:23.907066 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:23.935367 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:23.937741 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:23.937896 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:23.938423 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:23.939143 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:23.939198 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.939248 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:23.939273 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:23.947248 27781 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:43751
I20260807 08:33:23.947276 27967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:43751 every 8 connection(s)
I20260807 08:33:23.947994 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:23.954176 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 27781
I20260807 08:33:23.954285 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:23.954593 1653 external_mini_cluster.cc:1428] Setting key 5eac57487e7de3d72630e4e629b7e0c6
Aug 07 08:33:23 dist-test-slave-6zbq krb5kdc[27413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:23.959857 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:23.949496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:36839 (local address 127.1.157.126:45537)
0807 08:33:23.949642 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:23.949647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:23.950402 (+ 755us) server_negotiation.cc:407] Connection header received
0807 08:33:23.951236 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:23.951239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:23.951295 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:23.951366 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:23.952405 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.952921 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.953976 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.954200 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.954294 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:23.956565 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:23.956569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:23.956571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:23.956596 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:23.957950 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.958525 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.958527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.958528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.958599 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:23.958924 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:23.958926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:23.958927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:23.959078 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:23.959144 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:23.959657 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 08:33:23.959746 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:33:23.960772 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:23.949848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45537 (local address 127.1.157.66:36839)
0807 08:33:23.950258 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:23.950296 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:23.951022 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:23.951528 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:23.951534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:23.951842 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:23.952272 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:23.952285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.953031 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.953034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:23.953862 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:23.953871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:23.954376 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:23.954382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:23.954570 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:23.955082 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:23.955097 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:23.956419 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:23.958076 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:23.958082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:23.958096 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:23.958442 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:23.958680 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:23.958683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:23.958685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:23.958847 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:23.959162 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:23.959168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:23.959441 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 08:33:23.959612 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 08:33:23.959901 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260807 08:33:23.961786 27969 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45537
I20260807 08:33:23.961978 27969 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:23.962421 27969 heartbeater.cc:511] Master 127.1.157.126:45537 requested a full tablet report, sending...
I20260807 08:33:23.963305 27484 ts_manager.cc:194] Registered new tserver with Master: e8b52e2f76ab4359b87456a0d1c850ef (127.1.157.66:43751)
I20260807 08:33:23.963820 27484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:36839
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:33:23.978932 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:44073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:24.082098 27975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:24.082316 27975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:24.082384 27975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:24.086193 27975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:24.086241 27975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:24.086295 27975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:24.091146 27975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:24.091926 27975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:24.092730 27975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:24.101007 27986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.101032 27984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.101032 27983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:24.101121 27975 server_base.cc:1034] running on GCE node
I20260807 08:33:24.101533 27975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:24.102054 27975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:24.103196 27975 hybrid_clock.cc:648] HybridClock initialized: now 1786091604103184 us; error 22 us; skew 500 ppm
Aug 07 08:33:24 dist-test-slave-6zbq krb5kdc[27413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:24.105589 27975 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:33:24.106324 27975 webserver.cc:460] Webserver started at http://127.1.157.67:37029/ using document root <none> and password file <none>
I20260807 08:33:24.106829 27975 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:24.106889 27975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:24.107062 27975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:24.108840 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "0e1eb4cd6b7644f6992083439696e280"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "ad53acfa212a87ca5ae8cb953c7e0265"
server_key_iv: "162f96424da6a10c9f7048110fcadac3"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.109324 27975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "0e1eb4cd6b7644f6992083439696e280"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "ad53acfa212a87ca5ae8cb953c7e0265"
server_key_iv: "162f96424da6a10c9f7048110fcadac3"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.112586 27975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:33:24.114684 27995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.115761 27975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:24.115854 27975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "0e1eb4cd6b7644f6992083439696e280"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "ad53acfa212a87ca5ae8cb953c7e0265"
server_key_iv: "162f96424da6a10c9f7048110fcadac3"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.115938 27975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:24.142321 27975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:24.144779 27975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:24.144930 27975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:24.145429 27975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:24.146147 27975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:24.146196 27975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.146245 27975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:24.146271 27975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.154059 27975 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:39153
I20260807 08:33:24.154088 28164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:39153 every 8 connection(s)
I20260807 08:33:24.154860 27975 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:33:24 dist-test-slave-6zbq krb5kdc[27413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:24.163837 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 27975
I20260807 08:33:24.163967 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:24.164285 1653 external_mini_cluster.cc:1428] Setting key 877986d00b00ade070c2e1bf1654284f
I20260807 08:33:24.166759 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.156370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:45187 (local address 127.1.157.126:45537)
0807 08:33:24.156494 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.156498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.157218 (+ 720us) server_negotiation.cc:407] Connection header received
0807 08:33:24.158035 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.158039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.158098 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.158168 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:24.159250 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.159773 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.160838 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.161056 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.161153 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.163355 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:24.163358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:24.163360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:24.163384 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:24.164766 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:24.165304 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:24.165307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:24.165309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:24.165404 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:24.165741 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:24.165743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:24.165744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:24.165913 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:24.165986 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.166541 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.166637 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:33:24.167403 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.156647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45537 (local address 127.1.157.67:45187)
0807 08:33:24.157067 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.157105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.157840 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.158362 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.158369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.158685 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:24.159106 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.159119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.159885 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.159888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.160723 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.160732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.161203 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.161207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.161358 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.161814 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:24.161835 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:24.163201 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:24.164865 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:24.164870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:24.164882 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:24.165210 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:24.165490 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:24.165493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:24.165495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:24.165648 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:24.166003 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:24.166009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:24.166284 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.166478 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.166712 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260807 08:33:24.168399 28165 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45537
I20260807 08:33:24.168586 28165 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:24.168968 28165 heartbeater.cc:511] Master 127.1.157.126:45537 requested a full tablet report, sending...
I20260807 08:33:24.169836 27484 ts_manager.cc:194] Registered new tserver with Master: 0e1eb4cd6b7644f6992083439696e280 (127.1.157.67:39153)
I20260807 08:33:24.170379 27484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:45187
I20260807 08:33:24.177588 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:24.185173 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.179368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.1.157.126:45537)
0807 08:33:24.179469 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.179472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.179632 (+ 160us) server_negotiation.cc:407] Connection header received
0807 08:33:24.179720 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.179722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.179755 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.179814 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:24.180390 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.180856 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.181602 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.181800 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.181861 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.182942 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:24.182945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:24.182947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:24.182968 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:24.183996 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:24.184343 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:24.184345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:24.184346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:24.184404 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:24.184575 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:24.184576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:24.184577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:24.184757 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:24.184851 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.184993 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.185072 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:33:24.187520 27484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:24.190264 27484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:24.202803 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.197800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:36669 (local address 127.0.0.1:38016)
0807 08:33:24.198123 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.198182 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.198312 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.199038 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.199044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.199068 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:24.199207 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.199214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.200312 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.200316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.201201 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.201212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.202228 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.202231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.202423 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.202463 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.202562 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.202675 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":49,"thread_start_us":85,"threads_started":1}
I20260807 08:33:24.202843 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.197917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.1.157.65:36669)
0807 08:33:24.198299 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.198310 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.198357 (+ 47us) server_negotiation.cc:407] Connection header received
0807 08:33:24.198480 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.198486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.198690 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.198839 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:24.199346 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.200206 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.201329 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.202107 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.202218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.202393 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.202608 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.202701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":139,"threads_started":1}
I20260807 08:33:24.204010 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.198827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:43751 (local address 127.0.0.1:56936)
0807 08:33:24.199373 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.199386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.199464 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.200024 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.200027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.200044 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:24.200129 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.200134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.201542 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.201545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.202601 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.202610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.203624 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.203628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.203731 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.203752 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.203825 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.203891 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":12,"thread_start_us":87,"threads_started":1}
I20260807 08:33:24.204059 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.199349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.1.157.66:43751)
0807 08:33:24.199606 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.199617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.199639 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:33:24.199710 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.199716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.199887 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.200042 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:24.200222 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.201383 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.202723 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.203518 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.203612 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.203741 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.203854 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.203935 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260807 08:33:24.204442 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.198503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39153 (local address 127.0.0.1:53876)
0807 08:33:24.198905 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.198917 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.198996 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.200071 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.200074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.200092 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:24.200189 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.200196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.201956 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.201959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.203074 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.203094 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.204087 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.204091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.204180 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.204202 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.204260 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.204323 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":59,"thread_start_us":98,"threads_started":1}
I20260807 08:33:24.205009 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.199195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.1.157.67:39153)
0807 08:33:24.199717 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.199728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.199745 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:33:24.199797 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.199801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.199932 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.200329 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:24.200453 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.201822 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.203197 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.203950 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.204631 (+ 681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.204754 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.204801 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.204880 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":88,"threads_started":1}
I20260807 08:33:24.205994 27669 tablet_service.cc:1467] Processing CreateTablet for tablet db01fa35568e4926aadc45e349636476 (DEFAULT_TABLE table=test-table [id=728bab3b1fd7483fa047a5866fd845be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:24.206691 27880 tablet_service.cc:1467] Processing CreateTablet for tablet db01fa35568e4926aadc45e349636476 (DEFAULT_TABLE table=test-table [id=728bab3b1fd7483fa047a5866fd845be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:24.207134 27669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db01fa35568e4926aadc45e349636476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:24.207489 27880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db01fa35568e4926aadc45e349636476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:24.207599 28061 tablet_service.cc:1467] Processing CreateTablet for tablet db01fa35568e4926aadc45e349636476 (DEFAULT_TABLE table=test-table [id=728bab3b1fd7483fa047a5866fd845be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:24.208515 28061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db01fa35568e4926aadc45e349636476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:24.213199 28188 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Bootstrap starting.
I20260807 08:33:24.214428 28189 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Bootstrap starting.
I20260807 08:33:24.215417 28190 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Bootstrap starting.
I20260807 08:33:24.216502 28188 tablet_bootstrap.cc:654] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:24.216686 28189 tablet_bootstrap.cc:654] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:24.217108 28190 tablet_bootstrap.cc:654] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:24.217386 28188 log.cc:824] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:24.217829 28189 log.cc:824] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:24.218115 28190 log.cc:824] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:24.219561 28188 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: No bootstrap required, opened a new log
I20260807 08:33:24.219585 28189 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: No bootstrap required, opened a new log
I20260807 08:33:24.219731 28188 ts_tablet_manager.cc:1397] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 08:33:24.219745 28189 ts_tablet_manager.cc:1397] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:24.222437 28189 raft_consensus.cc:348] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.222453 28188 raft_consensus.cc:348] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.222637 28188 raft_consensus.cc:374] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:24.222643 28189 raft_consensus.cc:374] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:24.222694 28188 raft_consensus.cc:729] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b52e2f76ab4359b87456a0d1c850ef, State: Initialized, Role: FOLLOWER
I20260807 08:33:24.222695 28189 raft_consensus.cc:729] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f862ca5287754128aeb9e4c823310acf, State: Initialized, Role: FOLLOWER
I20260807 08:33:24.223065 28188 consensus_queue.cc:260] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.223066 28189 consensus_queue.cc:260] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.223620 27969 heartbeater.cc:503] Master 127.1.157.126:45537 was elected leader, sending a full tablet report...
I20260807 08:33:24.223631 27776 heartbeater.cc:503] Master 127.1.157.126:45537 was elected leader, sending a full tablet report...
I20260807 08:33:24.223896 28189 ts_tablet_manager.cc:1428] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260807 08:33:24.225106 28188 ts_tablet_manager.cc:1428] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:33:24.226783 28190 tablet_bootstrap.cc:492] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: No bootstrap required, opened a new log
I20260807 08:33:24.226958 28190 ts_tablet_manager.cc:1397] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260807 08:33:24.229820 28190 raft_consensus.cc:348] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.230002 28190 raft_consensus.cc:374] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:24.230046 28190 raft_consensus.cc:729] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1eb4cd6b7644f6992083439696e280, State: Initialized, Role: FOLLOWER
I20260807 08:33:24.230324 28190 consensus_queue.cc:260] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.230895 28165 heartbeater.cc:503] Master 127.1.157.126:45537 was elected leader, sending a full tablet report...
I20260807 08:33:24.231163 28190 ts_tablet_manager.cc:1428] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260807 08:33:24.239359 27777 tablet.cc:2369] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:24.319526 28201 raft_consensus.cc:482] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:24.319743 28201 raft_consensus.cc:504] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.321051 28201 leader_election.cc:288] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1eb4cd6b7644f6992083439696e280 (127.1.157.67:39153), e8b52e2f76ab4359b87456a0d1c850ef (127.1.157.66:43751)
I20260807 08:33:24.324713 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.321407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:56523 (local address 127.1.157.67:39153)
0807 08:33:24.321532 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.321536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.321585 (+ 49us) server_negotiation.cc:407] Connection header received
0807 08:33:24.321751 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.321754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.321805 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.321885 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:24.322156 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.322700 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.323812 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.324204 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.324315 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.324369 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.324532 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.324603 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:33:24.324729 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.321262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:39153 (local address 127.1.157.65:56523)
0807 08:33:24.321514 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.321529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.321638 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.321900 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.321903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.321922 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:24.322044 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.322049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.322807 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.322811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.323695 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.323704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.324315 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.324319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.324416 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.324441 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.324507 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.324578 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":37,"thread_start_us":86,"threads_started":1}
I20260807 08:33:24.324880 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.321746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:54379 (local address 127.1.157.66:43751)
0807 08:33:24.321875 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.321879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.321961 (+ 82us) server_negotiation.cc:407] Connection header received
0807 08:33:24.322065 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.322068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.322113 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.322180 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:24.322461 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.322943 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.323966 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.324357 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.324458 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.324509 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.324685 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.324751 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:33:24.325057 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.321607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:43751 (local address 127.1.157.65:54379)
0807 08:33:24.321880 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.321892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.321984 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.322205 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.322207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.322222 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:24.322357 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.322362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.323048 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.323051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.323851 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.323861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.324459 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.324462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.324569 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.324591 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.324652 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.324718 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":18,"thread_start_us":91,"threads_started":1}
I20260807 08:33:24.325353 28094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db01fa35568e4926aadc45e349636476" candidate_uuid: "f862ca5287754128aeb9e4c823310acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1eb4cd6b7644f6992083439696e280" is_pre_election: true
I20260807 08:33:24.325589 27906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db01fa35568e4926aadc45e349636476" candidate_uuid: "f862ca5287754128aeb9e4c823310acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" is_pre_election: true
I20260807 08:33:24.325629 28094 raft_consensus.cc:2454] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f862ca5287754128aeb9e4c823310acf in term 0.
I20260807 08:33:24.325841 27906 raft_consensus.cc:2454] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f862ca5287754128aeb9e4c823310acf in term 0.
I20260807 08:33:24.326067 27596 leader_election.cc:302] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1eb4cd6b7644f6992083439696e280, f862ca5287754128aeb9e4c823310acf; no voters:
I20260807 08:33:24.326363 28201 raft_consensus.cc:2781] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:24.326455 28201 raft_consensus.cc:482] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:24.326507 28201 raft_consensus.cc:3037] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:24.327489 28201 raft_consensus.cc:504] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.327768 28201 leader_election.cc:288] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [CANDIDATE]: Term 1 election: Requested vote from peers 0e1eb4cd6b7644f6992083439696e280 (127.1.157.67:39153), e8b52e2f76ab4359b87456a0d1c850ef (127.1.157.66:43751)
I20260807 08:33:24.328102 28094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db01fa35568e4926aadc45e349636476" candidate_uuid: "f862ca5287754128aeb9e4c823310acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1eb4cd6b7644f6992083439696e280"
I20260807 08:33:24.328153 27906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db01fa35568e4926aadc45e349636476" candidate_uuid: "f862ca5287754128aeb9e4c823310acf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b52e2f76ab4359b87456a0d1c850ef"
I20260807 08:33:24.328224 28094 raft_consensus.cc:3037] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:24.328256 27906 raft_consensus.cc:3037] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:24.329180 28094 raft_consensus.cc:2454] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f862ca5287754128aeb9e4c823310acf in term 1.
I20260807 08:33:24.329248 27906 raft_consensus.cc:2454] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f862ca5287754128aeb9e4c823310acf in term 1.
I20260807 08:33:24.329444 27596 leader_election.cc:302] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e1eb4cd6b7644f6992083439696e280, f862ca5287754128aeb9e4c823310acf; no voters:
I20260807 08:33:24.329627 28201 raft_consensus.cc:2781] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:24.329806 28201 raft_consensus.cc:686] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [term 1 LEADER]: Becoming Leader. State: Replica: f862ca5287754128aeb9e4c823310acf, State: Running, Role: LEADER
I20260807 08:33:24.330082 28201 consensus_queue.cc:237] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } }
I20260807 08:33:24.332885 27484 catalog_manager.cc:5495] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf reported cstate change: term changed from 0 to 1, leader changed from <none> to f862ca5287754128aeb9e4c823310acf (127.1.157.65). New cstate: current_term: 1 leader_uuid: "f862ca5287754128aeb9e4c823310acf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f862ca5287754128aeb9e4c823310acf" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 } health_report { overall_health: UNKNOWN } } }
I20260807 08:33:24.356797 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.353538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.1.157.65:36669)
0807 08:33:24.353663 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.353678 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.353691 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:33:24.353760 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.353763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.353812 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.353898 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:24.354122 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.354652 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.355480 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.355695 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.355781 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.356054 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:24.356421 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:24.356524 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.356611 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.356676 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:33:24.362239 27906 raft_consensus.cc:1261] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 1 FOLLOWER]: Refusing update from remote peer f862ca5287754128aeb9e4c823310acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:24.362238 28094 raft_consensus.cc:1261] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 1 FOLLOWER]: Refusing update from remote peer f862ca5287754128aeb9e4c823310acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:24.362917 28208 consensus_queue.cc:1035] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1eb4cd6b7644f6992083439696e280" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 39153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:24.363121 28201 consensus_queue.cc:1035] T db01fa35568e4926aadc45e349636476 P f862ca5287754128aeb9e4c823310acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b52e2f76ab4359b87456a0d1c850ef" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:24.369124 28212 mvcc.cc:205] Tried to move back new op lower bound from 7315831211460726784 to 7315831211338424320. Current Snapshot: MvccSnapshot[applied={T|T < 7315831211460726784}]
I20260807 08:33:24.369808 28218 mvcc.cc:205] Tried to move back new op lower bound from 7315831211460726784 to 7315831211338424320. Current Snapshot: MvccSnapshot[applied={T|T < 7315831211460726784}]
I20260807 08:33:24.371202 28215 mvcc.cc:205] Tried to move back new op lower bound from 7315831211460726784 to 7315831211338424320. Current Snapshot: MvccSnapshot[applied={T|T < 7315831211460726784}]
I20260807 08:33:24.377642 27483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:24.377789 27483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:24.380000 27483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0201ed0e4d094e8f999ad2c1f76a93e7: Sending DeleteTablet for 3 replicas of tablet db01fa35568e4926aadc45e349636476
I20260807 08:33:24.380818 27669 tablet_service.cc:1514] Processing DeleteTablet for tablet db01fa35568e4926aadc45e349636476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:24 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:38016
I20260807 08:33:24.380807 27880 tablet_service.cc:1514] Processing DeleteTablet for tablet db01fa35568e4926aadc45e349636476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:24 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:56936
I20260807 08:33:24.380931 28061 tablet_service.cc:1514] Processing DeleteTablet for tablet db01fa35568e4926aadc45e349636476 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:24 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:53876
I20260807 08:33:24.381211 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 27573
I20260807 08:33:24.381336 28231 tablet_replica.cc:330] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: stopping tablet replica
I20260807 08:33:24.381614 28231 raft_consensus.cc:2229] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:24.381917 28231 raft_consensus.cc:2258] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:24.383694 28229 tablet_replica.cc:330] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: stopping tablet replica
I20260807 08:33:24.383971 28229 raft_consensus.cc:2229] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:24.384210 28229 raft_consensus.cc:2258] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:24.386087 28231 ts_tablet_manager.cc:1905] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:24.386946 28229 ts_tablet_manager.cc:1905] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:24.388794 28231 ts_tablet_manager.cc:1918] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:24.388886 28231 log.cc:1196] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/db01fa35568e4926aadc45e349636476
I20260807 08:33:24.389199 28231 ts_tablet_manager.cc:1939] T db01fa35568e4926aadc45e349636476 P 0e1eb4cd6b7644f6992083439696e280: Deleting consensus metadata
I20260807 08:33:24.389411 28229 ts_tablet_manager.cc:1918] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:24.389504 28229 log.cc:1196] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/db01fa35568e4926aadc45e349636476
I20260807 08:33:24.389799 28229 ts_tablet_manager.cc:1939] T db01fa35568e4926aadc45e349636476 P e8b52e2f76ab4359b87456a0d1c850ef: Deleting consensus metadata
I20260807 08:33:24.390414 27464 catalog_manager.cc:4841] TS 0e1eb4cd6b7644f6992083439696e280 (127.1.157.67:39153): tablet db01fa35568e4926aadc45e349636476 (table test-table [id=728bab3b1fd7483fa047a5866fd845be]) successfully deleted
I20260807 08:33:24.390856 27464 catalog_manager.cc:4841] TS e8b52e2f76ab4359b87456a0d1c850ef (127.1.157.66:43751): tablet db01fa35568e4926aadc45e349636476 (table test-table [id=728bab3b1fd7483fa047a5866fd845be]) successfully deleted
W20260807 08:33:24.392263 27464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:36669 (error 108)
I20260807 08:33:24.392421 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 27781
I20260807 08:33:24.397962 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.397667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:36669 (local address 127.0.0.1:38032)
0807 08:33:24.397806 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.397881 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:33:24.398136 27464 catalog_manager.cc:4584] TS f862ca5287754128aeb9e4c823310acf (127.1.157.65:36669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db01fa35568e4926aadc45e349636476: Network error: Client connection negotiation failed: client connection to 127.1.157.65:36669: connect: Connection refused (error 111)
I20260807 08:33:24.402088 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 27975
I20260807 08:33:24.412708 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 27433
2026-08-07T08:33:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2371 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T08:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:24Z Disabled control of system clock
I20260807 08:33:24.445066 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34385
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:34385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:24.548821 28239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:24.549033 28239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:24.549070 28239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:24.553158 28239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:24.553213 28239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:24.553227 28239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:24.553242 28239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:24.553256 28239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:24.557866 28239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:34385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:34385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:24.558733 28239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:24.559989 28239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:24.567667 28244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.567734 28246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.567684 28249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:24.567833 28239 server_base.cc:1034] running on GCE node
I20260807 08:33:24.568380 28239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:24.569289 28239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:24.570452 28239 hybrid_clock.cc:648] HybridClock initialized: now 1786091604570431 us; error 35 us; skew 500 ppm
I20260807 08:33:24.572057 28239 webserver.cc:460] Webserver started at http://127.1.157.126:35575/ using document root <none> and password file <none>
I20260807 08:33:24.572544 28239 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:24.572608 28239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:24.572788 28239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:24.574666 28239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "9572b1322d1a470a9fa97ba16d70b78f"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "720186326c202038ffb2a7c7e76b9b89"
server_key_iv: "fe272b3cfe5e049110d9abb1a6aee01a"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.575181 28239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "9572b1322d1a470a9fa97ba16d70b78f"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "720186326c202038ffb2a7c7e76b9b89"
server_key_iv: "fe272b3cfe5e049110d9abb1a6aee01a"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.578444 28239 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:24.580559 28255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.581607 28239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:24.581723 28239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "9572b1322d1a470a9fa97ba16d70b78f"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "720186326c202038ffb2a7c7e76b9b89"
server_key_iv: "fe272b3cfe5e049110d9abb1a6aee01a"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.581809 28239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:24.602421 28239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:24.602990 28239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:24.603147 28239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:24.610885 28239 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:34385
I20260807 08:33:24.610889 28325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:34385 every 8 connection(s)
I20260807 08:33:24.611737 28239 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:24.614576 28326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:24.619692 28326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: Bootstrap starting.
I20260807 08:33:24.620046 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 28239
I20260807 08:33:24.620167 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:24.620399 1653 external_mini_cluster.cc:1428] Setting key 582bac18460a0a12d5988dedcd41b1a3
I20260807 08:33:24.622038 28326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:24.622766 28326 log.cc:824] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:24.624619 28326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: No bootstrap required, opened a new log
I20260807 08:33:24.628500 28326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } }
I20260807 08:33:24.628723 28326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:24.628779 28326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9572b1322d1a470a9fa97ba16d70b78f, State: Initialized, Role: FOLLOWER
I20260807 08:33:24.629112 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.621663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.1.157.126:34385)
0807 08:33:24.622252 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.622266 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.622299 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:24.622964 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.622984 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.623346 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.623608 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:24.623914 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.624887 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.626069 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.626558 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.626678 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.627452 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:24.627456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:24.627467 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:24.627820 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:24.627873 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:24.627892 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:24.627963 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.628091 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.628337 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":105,"threads_started":1}
I20260807 08:33:24.629142 28326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } }
I20260807 08:33:24.629295 28326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:24.629350 28326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:24.629446 28326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:24.630208 28326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } }
I20260807 08:33:24.630529 28326 leader_election.cc:302] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9572b1322d1a470a9fa97ba16d70b78f; no voters:
I20260807 08:33:24.630787 28326 leader_election.cc:288] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:24.630921 28337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:24.631139 28337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [term 1 LEADER]: Becoming Leader. State: Replica: 9572b1322d1a470a9fa97ba16d70b78f, State: Running, Role: LEADER
I20260807 08:33:24.631430 28337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } }
I20260807 08:33:24.631932 28326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:24.632517 28337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9572b1322d1a470a9fa97ba16d70b78f. Latest consensus state: current_term: 1 leader_uuid: "9572b1322d1a470a9fa97ba16d70b78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } } }
I20260807 08:33:24.632635 28337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:24.633001 28339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9572b1322d1a470a9fa97ba16d70b78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9572b1322d1a470a9fa97ba16d70b78f" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 34385 } } }
I20260807 08:33:24.633108 28339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:24.636636 28356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:24.636713 28356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:24.636781 28345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:24.637462 28345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:24.641985 28345 catalog_manager.cc:1347] Generated new cluster ID: 16ac54057ee648a1996e72b7c476642e
I20260807 08:33:24.642032 28345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:24.659504 28345 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:24.660561 28345 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:24.665050 28345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: Generated new TSK 0
I20260807 08:33:24.665696 28345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:24.674510 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--builtin_ntp_servers=127.1.157.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:24.799114 28362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:24.799340 28362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:24.799376 28362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:24.803277 28362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:24.803321 28362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:24.803371 28362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:24.807824 28362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:24.808691 28362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:24.809880 28362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:24.818650 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.818657 28369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:24.818684 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:24.818696 28362 server_base.cc:1034] running on GCE node
I20260807 08:33:24.819221 28362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:24.819751 28362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:24.820921 28362 hybrid_clock.cc:648] HybridClock initialized: now 1786091604820903 us; error 28 us; skew 500 ppm
I20260807 08:33:24.822568 28362 webserver.cc:460] Webserver started at http://127.1.157.65:40977/ using document root <none> and password file <none>
I20260807 08:33:24.823053 28362 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:24.823110 28362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:24.823289 28362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:24.825119 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "d3cea5393de443098c19ac9b8e0b0aa1"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "6fd54943e7c08ad9275d95a6c34f3b84"
server_key_iv: "5f88e39b5ab40c34dcd3a33e3456cede"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.825630 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "d3cea5393de443098c19ac9b8e0b0aa1"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "6fd54943e7c08ad9275d95a6c34f3b84"
server_key_iv: "5f88e39b5ab40c34dcd3a33e3456cede"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.828882 28362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:33:24.831049 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.832180 28362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:24.832283 28362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "d3cea5393de443098c19ac9b8e0b0aa1"
format_stamp: "Formatted at 2026-08-07 08:33:24 on dist-test-slave-6zbq"
server_key: "6fd54943e7c08ad9275d95a6c34f3b84"
server_key_iv: "5f88e39b5ab40c34dcd3a33e3456cede"
server_key_version: "encryptionkey@0"
I20260807 08:33:24.832371 28362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:24.859019 28362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:24.859575 28362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:24.859725 28362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:24.860225 28362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:24.861049 28362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:24.861106 28362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.861156 28362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:24.861193 28362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:24.869240 28362 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:38227
I20260807 08:33:24.869279 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:38227 every 8 connection(s)
I20260807 08:33:24.870025 28362 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:24.871496 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 28362
I20260807 08:33:24.871595 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:24.871868 1653 external_mini_cluster.cc:1428] Setting key 45ff6369cdeaa0f30d77bf8ce96511ae
I20260807 08:33:24.873796 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--builtin_ntp_servers=127.1.157.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:24.879341 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.871866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:46495 (local address 127.1.157.126:34385)
0807 08:33:24.872012 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:24.872017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:24.873138 (+ 1121us) server_negotiation.cc:407] Connection header received
0807 08:33:24.874861 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:24.874863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:24.874907 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:24.874973 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:24.875983 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.876516 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.877594 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.877819 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.877905 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.878896 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:24.878900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:24.878903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:24.878916 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:24.878935 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:24.878943 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:24.879000 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:24.879135 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:33:24.879201 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 08:33:24.880067 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:24.872276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34385 (local address 127.1.157.65:46495)
0807 08:33:24.872995 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:24.873045 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:24.874639 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:24.875137 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:24.875145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:24.875245 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:24.875845 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.875860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.876634 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.876637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.877477 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:24.877486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:24.877950 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:24.877954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:24.878108 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:24.878654 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:24.878656 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:24.878700 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:24.878758 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:24.879010 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:24.879021 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:24.879026 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 08:33:24.879126 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 08:33:24.879375 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":131,"threads_started":1}
I20260807 08:33:24.881301 28562 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34385
I20260807 08:33:24.881505 28562 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:24.881948 28562 heartbeater.cc:511] Master 127.1.157.126:34385 requested a full tablet report, sending...
I20260807 08:33:24.883510 28271 ts_manager.cc:194] Registered new tserver with Master: d3cea5393de443098c19ac9b8e0b0aa1 (127.1.157.65:38227)
I20260807 08:33:24.885100 28271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.65:46495
W20260807 08:33:24.982967 28565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:24.983177 28565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:24.983213 28565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:24.987139 28565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:24.987185 28565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:24.987246 28565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:24.991767 28565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:24.992619 28565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:24.993809 28565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:25.002513 28579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:25.002521 28576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:25.002594 28575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:25.002736 28565 server_base.cc:1034] running on GCE node
I20260807 08:33:25.003103 28565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:25.003654 28565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:25.004802 28565 hybrid_clock.cc:648] HybridClock initialized: now 1786091605004779 us; error 36 us; skew 500 ppm
I20260807 08:33:25.006412 28565 webserver.cc:460] Webserver started at http://127.1.157.66:36105/ using document root <none> and password file <none>
I20260807 08:33:25.006923 28565 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:25.006985 28565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:25.007167 28565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:25.008992 28565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "0f3ecb94059046c4b61a61c8d86da95d"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "9f1f2ccc413fe7bf5c4046923a91dfb5"
server_key_iv: "e83af8d9365fe11a35f3c4d7e240a4bc"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.009498 28565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "0f3ecb94059046c4b61a61c8d86da95d"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "9f1f2ccc413fe7bf5c4046923a91dfb5"
server_key_iv: "e83af8d9365fe11a35f3c4d7e240a4bc"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.012912 28565 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:33:25.015172 28592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.016260 28565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:33:25.016350 28565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "0f3ecb94059046c4b61a61c8d86da95d"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "9f1f2ccc413fe7bf5c4046923a91dfb5"
server_key_iv: "e83af8d9365fe11a35f3c4d7e240a4bc"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.016427 28565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:25.045744 28565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:25.046255 28565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:25.046427 28565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:25.046914 28565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:25.047722 28565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:25.047766 28565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.047801 28565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:25.047820 28565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.055647 28565 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:38235
I20260807 08:33:25.055672 28756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:38235 every 8 connection(s)
I20260807 08:33:25.056404 28565 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:25.060451 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 28565
I20260807 08:33:25.060563 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:25.060807 1653 external_mini_cluster.cc:1428] Setting key b53506e66b15cd95766a6cb810bbf59f
I20260807 08:33:25.062590 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--builtin_ntp_servers=127.1.157.84:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:25.065012 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.057957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57595 (local address 127.1.157.126:34385)
0807 08:33:25.058077 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.058081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.059063 (+ 982us) server_negotiation.cc:407] Connection header received
0807 08:33:25.060374 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.060377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.060437 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.060507 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:25.061444 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.061968 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.063202 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.063469 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.063555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.064611 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:25.064615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:25.064617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:25.064628 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:25.064650 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:25.064658 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:25.064718 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.064858 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.064906 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:33:25.066075 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.058266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34385 (local address 127.1.157.66:57595)
0807 08:33:25.058919 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.058957 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.060180 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.060687 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.060694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.060772 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:25.061308 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.061329 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.062090 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.062100 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.063053 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.063063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.063617 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.063621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.063769 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.064336 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:25.064338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:25.064389 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:25.064465 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:25.064732 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:25.064739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:25.064745 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.064851 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.065210 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
I20260807 08:33:25.067163 28757 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34385
I20260807 08:33:25.067358 28757 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:25.067791 28757 heartbeater.cc:511] Master 127.1.157.126:34385 requested a full tablet report, sending...
I20260807 08:33:25.068727 28271 ts_manager.cc:194] Registered new tserver with Master: 0f3ecb94059046c4b61a61c8d86da95d (127.1.157.66:38235)
I20260807 08:33:25.069252 28271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.66:57595
W20260807 08:33:25.169435 28760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:25.169658 28760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:25.169708 28760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:25.173606 28760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:25.173650 28760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:25.173718 28760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:25.178191 28760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:34385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:25.179085 28760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:25.180265 28760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:25.188846 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:25.188854 28767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:25.188923 28760 server_base.cc:1034] running on GCE node
W20260807 08:33:25.188854 28770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:25.189411 28760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:25.189950 28760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:25.191092 28760 hybrid_clock.cc:648] HybridClock initialized: now 1786091605191070 us; error 36 us; skew 500 ppm
I20260807 08:33:25.192688 28760 webserver.cc:460] Webserver started at http://127.1.157.67:39655/ using document root <none> and password file <none>
I20260807 08:33:25.193174 28760 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:25.193238 28760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:25.193415 28760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:25.195250 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "9a776473bc73440a8f8ab3a4e334d661"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "c6cdcddbead30e89c6277432ab4857bf"
server_key_iv: "6a6c06e1c8c5ba2e4a7c6114c85f73e0"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.195756 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "9a776473bc73440a8f8ab3a4e334d661"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "c6cdcddbead30e89c6277432ab4857bf"
server_key_iv: "6a6c06e1c8c5ba2e4a7c6114c85f73e0"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.199030 28760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:25.201112 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.202165 28760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:25.202266 28760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "9a776473bc73440a8f8ab3a4e334d661"
format_stamp: "Formatted at 2026-08-07 08:33:25 on dist-test-slave-6zbq"
server_key: "c6cdcddbead30e89c6277432ab4857bf"
server_key_iv: "6a6c06e1c8c5ba2e4a7c6114c85f73e0"
server_key_version: "encryptionkey@0"
I20260807 08:33:25.202378 28760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:25.219602 28760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:25.220181 28760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:25.220326 28760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:25.220813 28760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:25.221614 28760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:25.221673 28760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.221730 28760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:25.221753 28760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:25.229346 28760 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:41385
I20260807 08:33:25.229406 28957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:41385 every 8 connection(s)
I20260807 08:33:25.230104 28760 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:33:25.238180 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.231674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:42065 (local address 127.1.157.126:34385)
0807 08:33:25.231785 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.231789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.232778 (+ 989us) server_negotiation.cc:407] Connection header received
0807 08:33:25.234073 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.234075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.234130 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.234199 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:25.234981 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.235564 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.236645 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.236863 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.236935 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.237826 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:25.237829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:25.237831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:25.237842 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:25.237859 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:25.237866 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:25.237914 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.238012 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.238057 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:33:25.238924 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.232009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:34385 (local address 127.1.157.67:42065)
0807 08:33:25.232604 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.232639 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.233908 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.234332 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.234368 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.234449 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:25.234880 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.234892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.235664 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.235667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.236537 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.236546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.236975 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.236977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.237105 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.237609 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:25.237611 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:25.237657 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:25.237713 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:25.237923 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:25.237928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:25.237931 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.238008 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.238198 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260807 08:33:25.239467 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 28760
I20260807 08:33:25.239557 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:25.239822 1653 external_mini_cluster.cc:1428] Setting key ece7e7f1c0f924a3ec0d5e1881627d95
I20260807 08:33:25.239953 28960 heartbeater.cc:347] Connected to a master server at 127.1.157.126:34385
I20260807 08:33:25.240159 28960 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:25.240684 28960 heartbeater.cc:511] Master 127.1.157.126:34385 requested a full tablet report, sending...
I20260807 08:33:25.241663 28280 ts_manager.cc:194] Registered new tserver with Master: 9a776473bc73440a8f8ab3a4e334d661 (127.1.157.67:41385)
I20260807 08:33:25.242193 28280 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.67:42065
I20260807 08:33:25.253149 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:25.258294 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.254945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.1.157.126:34385)
0807 08:33:25.255049 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.255053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.255213 (+ 160us) server_negotiation.cc:407] Connection header received
0807 08:33:25.255338 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.255341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.255379 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.255437 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:25.255702 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.256185 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.256976 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.257187 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.257270 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.257930 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:25.257934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:25.257936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:25.257948 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:25.257965 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:25.257972 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:25.258028 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.258110 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.258172 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:33:25.260649 28280 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:25.263423 28280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:25.275640 28978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.270673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:41385 (local address 127.0.0.1:32974)
0807 08:33:25.270972 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.271013 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.271118 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.271572 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.271578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.271602 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:25.271718 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.271725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.273028 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.273032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.273998 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.274008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.275188 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.275191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.275306 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.275340 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.275432 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.275538 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260807 08:33:25.276057 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.270782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.1.157.67:41385)
0807 08:33:25.271079 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.271090 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.271112 (+ 22us) server_negotiation.cc:407] Connection header received
0807 08:33:25.271252 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.271255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.271419 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.271586 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:25.271859 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.272910 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.274117 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.274864 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.274959 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.275118 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.275808 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.275906 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":122,"threads_started":1}
I20260807 08:33:25.276770 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.271325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:38227 (local address 127.0.0.1:49550)
0807 08:33:25.272322 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.272334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.272424 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.272863 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.272866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.272887 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:25.272978 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.272984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.274332 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.274335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.275433 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.275441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.276413 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.276416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.276503 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.276526 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.276586 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.276646 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"mutex_wait_us":24,"thread_start_us":90,"threads_started":1}
I20260807 08:33:25.276872 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.271512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.1.157.65:38227)
0807 08:33:25.271939 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.271950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.272424 (+ 474us) server_negotiation.cc:407] Connection header received
0807 08:33:25.272530 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.272536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.272726 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.272872 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:25.273088 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.274224 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.275546 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.276288 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.276397 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.276562 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.276635 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.276718 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":135,"threads_started":1}
I20260807 08:33:25.277276 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.271606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:38235 (local address 127.0.0.1:39398)
0807 08:33:25.272608 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.272620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.272702 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.273168 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.273170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.273187 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:25.273295 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.273301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.274564 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.274567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.275583 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.275594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.276912 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.276915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.276997 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.277018 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.277089 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.277152 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":26,"thread_start_us":99,"threads_started":1}
I20260807 08:33:25.277408 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.271921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.1.157.66:38235)
0807 08:33:25.272211 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.272221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.272793 (+ 572us) server_negotiation.cc:407] Connection header received
0807 08:33:25.272872 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.272876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.273043 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.273191 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:25.273398 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.274468 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.275893 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.276586 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.276685 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.276849 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.277154 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.277261 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260807 08:33:25.279037 28860 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1049365d04cd598c601192b8abf7e (DEFAULT_TABLE table=test-table [id=c3a8a6894a924861ad42213e9478ac93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:25.279310 28455 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1049365d04cd598c601192b8abf7e (DEFAULT_TABLE table=test-table [id=c3a8a6894a924861ad42213e9478ac93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:25.280069 28455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1049365d04cd598c601192b8abf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:25.280108 28860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1049365d04cd598c601192b8abf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:25.279973 28669 tablet_service.cc:1467] Processing CreateTablet for tablet 67d1049365d04cd598c601192b8abf7e (DEFAULT_TABLE table=test-table [id=c3a8a6894a924861ad42213e9478ac93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:25.280884 28669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d1049365d04cd598c601192b8abf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:25.285996 28985 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: Bootstrap starting.
I20260807 08:33:25.287240 28986 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Bootstrap starting.
I20260807 08:33:25.289052 28986 tablet_bootstrap.cc:654] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:25.289052 28985 tablet_bootstrap.cc:654] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:25.289896 28985 log.cc:824] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:25.289896 28986 log.cc:824] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:25.290292 28987 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Bootstrap starting.
I20260807 08:33:25.291834 28986 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: No bootstrap required, opened a new log
I20260807 08:33:25.291836 28985 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: No bootstrap required, opened a new log
I20260807 08:33:25.292003 28987 tablet_bootstrap.cc:654] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:25.292024 28985 ts_tablet_manager.cc:1397] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:33:25.292135 28986 ts_tablet_manager.cc:1397] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:33:25.292831 28987 log.cc:824] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:25.294848 28987 tablet_bootstrap.cc:492] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: No bootstrap required, opened a new log
I20260807 08:33:25.295018 28987 ts_tablet_manager.cc:1397] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:25.296141 28985 raft_consensus.cc:348] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.296141 28986 raft_consensus.cc:348] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.296399 28985 raft_consensus.cc:374] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:25.296399 28986 raft_consensus.cc:374] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:25.296454 28986 raft_consensus.cc:729] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a776473bc73440a8f8ab3a4e334d661, State: Initialized, Role: FOLLOWER
I20260807 08:33:25.296454 28985 raft_consensus.cc:729] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cea5393de443098c19ac9b8e0b0aa1, State: Initialized, Role: FOLLOWER
I20260807 08:33:25.296887 28985 consensus_queue.cc:260] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.296887 28986 consensus_queue.cc:260] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.297566 28562 heartbeater.cc:503] Master 127.1.157.126:34385 was elected leader, sending a full tablet report...
I20260807 08:33:25.297859 28985 ts_tablet_manager.cc:1428] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 08:33:25.297859 28986 ts_tablet_manager.cc:1428] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:33:25.298170 28960 heartbeater.cc:503] Master 127.1.157.126:34385 was elected leader, sending a full tablet report...
I20260807 08:33:25.298084 28987 raft_consensus.cc:348] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.298321 28987 raft_consensus.cc:374] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:25.298395 28987 raft_consensus.cc:729] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3ecb94059046c4b61a61c8d86da95d, State: Initialized, Role: FOLLOWER
I20260807 08:33:25.298822 28987 consensus_queue.cc:260] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.299811 28987 ts_tablet_manager.cc:1428] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 08:33:25.300137 28757 heartbeater.cc:503] Master 127.1.157.126:34385 was elected leader, sending a full tablet report...
I20260807 08:33:25.301626 28993 raft_consensus.cc:482] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:25.301770 28993 raft_consensus.cc:504] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.302918 28993 leader_election.cc:288] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3cea5393de443098c19ac9b8e0b0aa1 (127.1.157.65:38227), 0f3ecb94059046c4b61a61c8d86da95d (127.1.157.66:38235)
I20260807 08:33:25.306577 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.303400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:48407 (local address 127.1.157.66:38235)
0807 08:33:25.303514 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.303518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.303617 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:33:25.303719 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.303721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.303757 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.303815 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:25.304084 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.304582 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.305641 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.306055 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.306145 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.306201 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.306383 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.306467 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:33:25.306593 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.303288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:38235 (local address 127.1.157.67:48407)
0807 08:33:25.303546 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.303557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.303633 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.303837 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.303840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.303857 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:25.303989 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.303996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.304688 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.304691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.305544 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.305554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.306162 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.306165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.306257 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.306280 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.306361 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.306429 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":4,"thread_start_us":86,"threads_started":1}
I20260807 08:33:25.306829 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.303598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:48327 (local address 127.1.157.65:38227)
0807 08:33:25.303718 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.303722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.303735 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:33:25.303774 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.303777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.303816 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.303887 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:25.304128 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.304602 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.305705 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.306083 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.306198 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.306244 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.306659 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.306731 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:33:25.306600 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.303081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:38227 (local address 127.1.157.67:48327)
0807 08:33:25.303216 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.303230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:25.303297 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:25.303914 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:25.303917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:25.303935 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:25.304024 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.304028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.304704 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.304707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.305582 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:25.305593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.306204 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.306208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:25.306309 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.306328 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:33:25.306423 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 08:33:25.306482 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":34}
I20260807 08:33:25.307291 28695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1049365d04cd598c601192b8abf7e" candidate_uuid: "9a776473bc73440a8f8ab3a4e334d661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f3ecb94059046c4b61a61c8d86da95d" is_pre_election: true
W20260807 08:33:25.307294 28758 tablet.cc:2369] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:25.307413 28483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1049365d04cd598c601192b8abf7e" candidate_uuid: "9a776473bc73440a8f8ab3a4e334d661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" is_pre_election: true
I20260807 08:33:25.307535 28695 raft_consensus.cc:2454] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a776473bc73440a8f8ab3a4e334d661 in term 0.
I20260807 08:33:25.307642 28483 raft_consensus.cc:2454] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a776473bc73440a8f8ab3a4e334d661 in term 0.
I20260807 08:33:25.307966 28780 leader_election.cc:302] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f3ecb94059046c4b61a61c8d86da95d, 9a776473bc73440a8f8ab3a4e334d661; no voters:
I20260807 08:33:25.308204 28993 raft_consensus.cc:2781] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:25.308274 28993 raft_consensus.cc:482] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:25.308301 28993 raft_consensus.cc:3037] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:25.309213 28993 raft_consensus.cc:504] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.309490 28993 leader_election.cc:288] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [CANDIDATE]: Term 1 election: Requested vote from peers d3cea5393de443098c19ac9b8e0b0aa1 (127.1.157.65:38227), 0f3ecb94059046c4b61a61c8d86da95d (127.1.157.66:38235)
I20260807 08:33:25.309793 28483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1049365d04cd598c601192b8abf7e" candidate_uuid: "9a776473bc73440a8f8ab3a4e334d661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3cea5393de443098c19ac9b8e0b0aa1"
I20260807 08:33:25.309913 28483 raft_consensus.cc:3037] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:25.309937 28695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d1049365d04cd598c601192b8abf7e" candidate_uuid: "9a776473bc73440a8f8ab3a4e334d661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f3ecb94059046c4b61a61c8d86da95d"
I20260807 08:33:25.310042 28695 raft_consensus.cc:3037] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:25.311152 28695 raft_consensus.cc:2454] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a776473bc73440a8f8ab3a4e334d661 in term 1.
I20260807 08:33:25.311152 28483 raft_consensus.cc:2454] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a776473bc73440a8f8ab3a4e334d661 in term 1.
I20260807 08:33:25.311427 28780 leader_election.cc:302] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a776473bc73440a8f8ab3a4e334d661, d3cea5393de443098c19ac9b8e0b0aa1; no voters:
I20260807 08:33:25.311574 28993 raft_consensus.cc:2781] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:25.311789 28993 raft_consensus.cc:686] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 1 LEADER]: Becoming Leader. State: Replica: 9a776473bc73440a8f8ab3a4e334d661, State: Running, Role: LEADER
I20260807 08:33:25.312040 28993 consensus_queue.cc:237] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } }
I20260807 08:33:25.314729 28271 catalog_manager.cc:5495] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a776473bc73440a8f8ab3a4e334d661 (127.1.157.67). New cstate: current_term: 1 leader_uuid: "9a776473bc73440a8f8ab3a4e334d661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a776473bc73440a8f8ab3a4e334d661" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 41385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 } health_report { overall_health: UNKNOWN } } }
I20260807 08:33:25.324899 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.321789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32982 (local address 127.1.157.67:41385)
0807 08:33:25.321888 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:25.321892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:25.321905 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:33:25.322057 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:25.322060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:25.322107 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:25.322194 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:25.322420 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.322877 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.323669 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:25.323876 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:25.323957 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:25.324225 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:25.324554 (+ 329us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:25.324649 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:25.324734 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 08:33:25.324777 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:33:25.329651 28483 raft_consensus.cc:1261] T 67d1049365d04cd598c601192b8abf7e P d3cea5393de443098c19ac9b8e0b0aa1 [term 1 FOLLOWER]: Refusing update from remote peer 9a776473bc73440a8f8ab3a4e334d661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:25.329653 28695 raft_consensus.cc:1261] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 1 FOLLOWER]: Refusing update from remote peer 9a776473bc73440a8f8ab3a4e334d661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:25.330273 28998 consensus_queue.cc:1035] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3cea5393de443098c19ac9b8e0b0aa1" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:25.330458 28993 consensus_queue.cc:1035] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f3ecb94059046c4b61a61c8d86da95d" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:25.337139 29005 mvcc.cc:205] Tried to move back new op lower bound from 7315831215424372736 to 7315831215360372736. Current Snapshot: MvccSnapshot[applied={T|T < 7315831215424372736}]
I20260807 08:33:25.337515 29007 mvcc.cc:205] Tried to move back new op lower bound from 7315831215424372736 to 7315831215360372736. Current Snapshot: MvccSnapshot[applied={T|T < 7315831215424372736}]
I20260807 08:33:25.340412 29008 mvcc.cc:205] Tried to move back new op lower bound from 7315831215424372736 to 7315831215360372736. Current Snapshot: MvccSnapshot[applied={T|T < 7315831215424372736}]
I20260807 08:33:25.345038 28271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:25.345197 28271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:25.347491 28271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9572b1322d1a470a9fa97ba16d70b78f: Sending DeleteTablet for 3 replicas of tablet 67d1049365d04cd598c601192b8abf7e
I20260807 08:33:25.348170 28860 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1049365d04cd598c601192b8abf7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:25 UTC) from {username='slave'} at 127.0.0.1:32974
I20260807 08:33:25.348222 28455 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1049365d04cd598c601192b8abf7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:25 UTC) from {username='slave'} at 127.0.0.1:49550
I20260807 08:33:25.348707 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 28362
I20260807 08:33:25.348830 28669 tablet_service.cc:1514] Processing DeleteTablet for tablet 67d1049365d04cd598c601192b8abf7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:25 UTC) from {username='slave'} at 127.0.0.1:39398
I20260807 08:33:25.349421 29022 tablet_replica.cc:330] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: stopping tablet replica
I20260807 08:33:25.349757 29022 raft_consensus.cc:2229] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:33:25.350090 29022 raft_consensus.cc:2258] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:25.353508 29022 ts_tablet_manager.cc:1905] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:25.355971 29022 ts_tablet_manager.cc:1918] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:25.356055 29022 log.cc:1196] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/67d1049365d04cd598c601192b8abf7e
I20260807 08:33:25.356349 29022 ts_tablet_manager.cc:1939] T 67d1049365d04cd598c601192b8abf7e P 9a776473bc73440a8f8ab3a4e334d661: Deleting consensus metadata
I20260807 08:33:25.357795 28258 catalog_manager.cc:4841] TS 9a776473bc73440a8f8ab3a4e334d661 (127.1.157.67:41385): tablet 67d1049365d04cd598c601192b8abf7e (table test-table [id=c3a8a6894a924861ad42213e9478ac93]) successfully deleted
I20260807 08:33:25.362552 29029 tablet_replica.cc:330] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: stopping tablet replica
I20260807 08:33:25.362811 29029 raft_consensus.cc:2229] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:25.363054 29029 raft_consensus.cc:2258] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:25.364410 29029 ts_tablet_manager.cc:1905] T 67d1049365d04cd598c601192b8abf7e P 0f3ecb94059046c4b61a61c8d86da95d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 08:33:25.365586 28258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:38227 (error 108)
I20260807 08:33:25.365715 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 28565
I20260807 08:33:25.368510 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.368261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:38227 (local address 127.0.0.1:49556)
0807 08:33:25.368380 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.368439 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:38227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 08:33:25.368638 28258 catalog_manager.cc:4584] TS d3cea5393de443098c19ac9b8e0b0aa1 (127.1.157.65:38227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67d1049365d04cd598c601192b8abf7e: Network error: Client connection negotiation failed: client connection to 127.1.157.65:38227: connect: Connection refused (error 111)
I20260807 08:33:25.375651 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 28760
I20260807 08:33:25.375686 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:25.375475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:38235 (local address 127.0.0.1:39400)
0807 08:33:25.375573 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:25.375617 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.66:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 08:33:25.384035 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 28239
2026-08-07T08:33:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (971 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:25 dist-test-slave-6zbq krb5kdc[29034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:25 dist-test-slave-6zbq krb5kdc[29034](info): set up 2 sockets
Aug 07 08:33:25 dist-test-slave-6zbq krb5kdc[29034](info): commencing operation
krb5kdc: starting...
W20260807 08:33:26.490762 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:26 dist-test-slave-6zbq krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:26.602074 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45097
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:45097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:26.706179 29053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:26.706423 29053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:26.706475 29053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:26.710247 29053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:26.710292 29053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:26.710310 29053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:26.710325 29053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:26.710358 29053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:26.715198 29053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:45097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:45097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:26.715994 29053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:26.716825 29053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:26.724372 29059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:26.724355 29058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:26.724360 29053 server_base.cc:1034] running on GCE node
W20260807 08:33:26.724344 29061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:26.725047 29053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:26.725932 29053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:26.727078 29053 hybrid_clock.cc:648] HybridClock initialized: now 1786091606727062 us; error 32 us; skew 500 ppm
Aug 07 08:33:26 dist-test-slave-6zbq krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:26.729615 29053 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:26.730413 29053 webserver.cc:460] Webserver started at http://127.1.157.126:43243/ using document root <none> and password file <none>
I20260807 08:33:26.730885 29053 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:26.730947 29053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:26.731112 29053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:26.732887 29053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "9db20bc579a243278331281f7f8cb09a"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "c40507c31cddc2ddfcffe6e7d65c2334"
server_key_iv: "2807c92186ef3988dd3e76abe29e14d3"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.733387 29053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "9db20bc579a243278331281f7f8cb09a"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "c40507c31cddc2ddfcffe6e7d65c2334"
server_key_iv: "2807c92186ef3988dd3e76abe29e14d3"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.736709 29053 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:33:26.738793 29070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:26.739849 29053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:26.739954 29053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "9db20bc579a243278331281f7f8cb09a"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "c40507c31cddc2ddfcffe6e7d65c2334"
server_key_iv: "2807c92186ef3988dd3e76abe29e14d3"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.740037 29053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:26.749864 29053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:26.752251 29053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:26.752400 29053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:26.758389 29053 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:45097
I20260807 08:33:26.758458 29152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:45097 every 8 connection(s)
I20260807 08:33:26.759218 29053 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:26.761688 29154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:26.766703 29154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: Bootstrap starting.
I20260807 08:33:26.766903 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 29053
I20260807 08:33:26.767019 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:26.767290 1653 external_mini_cluster.cc:1428] Setting key ee2f2de936f7e8f7d6d5cccdfc76091e
I20260807 08:33:26.768956 29154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:26.769603 29154 log.cc:824] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:26.771575 29154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: No bootstrap required, opened a new log
I20260807 08:33:26.775506 29154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } }
I20260807 08:33:26.775709 29154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:26.775774 29154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db20bc579a243278331281f7f8cb09a, State: Initialized, Role: FOLLOWER
Aug 07 08:33:26 dist-test-slave-6zbq krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:26.776240 29154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } }
I20260807 08:33:26.776415 29154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:26.776492 29154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:26.776628 29154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:26.777782 29154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } }
I20260807 08:33:26.778091 29154 leader_election.cc:302] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9db20bc579a243278331281f7f8cb09a; no voters:
I20260807 08:33:26.778406 29154 leader_election.cc:288] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:26.778486 29163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:26.778699 29163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [term 1 LEADER]: Becoming Leader. State: Replica: 9db20bc579a243278331281f7f8cb09a, State: Running, Role: LEADER
I20260807 08:33:26.779013 29163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } }
I20260807 08:33:26.779665 29154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:26.780764 29167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9db20bc579a243278331281f7f8cb09a. Latest consensus state: current_term: 1 leader_uuid: "9db20bc579a243278331281f7f8cb09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } } }
I20260807 08:33:26.780902 29167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:26.781069 29165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9db20bc579a243278331281f7f8cb09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9db20bc579a243278331281f7f8cb09a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 45097 } } }
I20260807 08:33:26.781262 29165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:26.781461 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:26.768461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.1.157.126:45097)
0807 08:33:26.768851 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:26.768866 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:26.768898 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:33:26.769508 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:26.769524 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:26.769845 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:26.770176 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:26.771009 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:26.772015 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:26.773167 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:26.773610 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:26.773722 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:26.776532 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:26.776543 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:26.776571 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:26.776616 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:26.778382 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:26.778848 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:26.778854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:26.778861 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:26.778997 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:26.779299 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:26.779302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:26.779302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:26.779656 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:26.779835 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:26.780155 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 08:33:26.780460 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":129,"threads_started":1}
I20260807 08:33:26.781514 29177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:26.784662 29177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:26.789767 29177 catalog_manager.cc:1347] Generated new cluster ID: 6d4c00ff6b504301a820054e252d5976
I20260807 08:33:26.789821 29177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:26.798003 29177 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:26.798836 29177 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:26.806089 29177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: Generated new TSK 0
I20260807 08:33:26.806711 29177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:33:26.834441 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:26.939373 29193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:26.939580 29193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:26.939635 29193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:26.943542 29193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:26.943589 29193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:26.943661 29193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:26.948524 29193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:26.949358 29193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:26.950194 29193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:26.958551 29204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:26.958557 29206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:26.958608 29193 server_base.cc:1034] running on GCE node
W20260807 08:33:26.958563 29208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:26.959045 29193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:26.959568 29193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:26.960737 29193 hybrid_clock.cc:648] HybridClock initialized: now 1786091606960720 us; error 31 us; skew 500 ppm
Aug 07 08:33:26 dist-test-slave-6zbq krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:26.963181 29193 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:33:26.963881 29193 webserver.cc:460] Webserver started at http://127.1.157.65:34523/ using document root <none> and password file <none>
I20260807 08:33:26.964329 29193 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:26.964383 29193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:26.964533 29193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:26.966287 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "b363ddafd4b74c26ac576b4e7259f7f2"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "0b5405204e63a3339b6f18561b13907c"
server_key_iv: "36a855bc4e23eae52d1ce0933b5a02d7"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.966806 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "b363ddafd4b74c26ac576b4e7259f7f2"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "0b5405204e63a3339b6f18561b13907c"
server_key_iv: "36a855bc4e23eae52d1ce0933b5a02d7"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.969991 29193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260807 08:33:26.972110 29217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:26.973170 29193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:26.973268 29193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "b363ddafd4b74c26ac576b4e7259f7f2"
format_stamp: "Formatted at 2026-08-07 08:33:26 on dist-test-slave-6zbq"
server_key: "0b5405204e63a3339b6f18561b13907c"
server_key_iv: "36a855bc4e23eae52d1ce0933b5a02d7"
server_key_version: "encryptionkey@0"
I20260807 08:33:26.973342 29193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:26.992302 29193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:26.994601 29193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:26.994727 29193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:26.995164 29193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:26.995836 29193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:26.995880 29193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:26.995913 29193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:26.995934 29193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.003718 29193 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:36995
I20260807 08:33:27.003785 29389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:36995 every 8 connection(s)
I20260807 08:33:27.004470 29193 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:27.009706 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 29193
I20260807 08:33:27.009835 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:27.010082 1653 external_mini_cluster.cc:1428] Setting key 217e2f0a64498919b145327c3139ba56
Aug 07 08:33:27 dist-test-slave-6zbq krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091606, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:27.016628 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.005964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:53319 (local address 127.1.157.126:45097)
0807 08:33:27.006107 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.006112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.006872 (+ 760us) server_negotiation.cc:407] Connection header received
0807 08:33:27.007950 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.007952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.007996 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.008063 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:27.009019 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.009622 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.010879 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.011117 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.011213 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.013503 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:27.013506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:27.013508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:27.013530 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:27.014847 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.015292 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.015295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.015296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.015373 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.015675 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.015678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.015680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.015842 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:27.015924 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.016355 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.016475 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:33:27.017269 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.006282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45097 (local address 127.1.157.65:53319)
0807 08:33:27.006743 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.006782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.007783 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.008192 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.008198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.008493 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:27.008909 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.008921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.009729 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.009732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.010760 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.010771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.011266 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.011270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.011446 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.011939 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:27.011954 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:27.013350 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:27.014954 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.014959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.014971 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.015199 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.015452 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.015455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.015457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.015590 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.015938 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:27.015944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:27.016146 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.016296 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.016563 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260807 08:33:27.018306 29391 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45097
I20260807 08:33:27.018553 29391 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:27.019042 29391 heartbeater.cc:511] Master 127.1.157.126:45097 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
I20260807 08:33:27.020555 29096 ts_manager.cc:194] Registered new tserver with Master: b363ddafd4b74c26ac576b4e7259f7f2 (127.1.157.65:36995)
I20260807 08:33:27.021603 29096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:53319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:33:27.034821 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:27.138842 29397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:27.139062 29397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:27.139119 29397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:27.143013 29397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:27.143064 29397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:27.143138 29397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:27.148069 29397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:27.148940 29397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:27.149817 29397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:27.158265 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:27.158299 29411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.158399 29397 server_base.cc:1034] running on GCE node
W20260807 08:33:27.158303 29407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.158880 29397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:27.159437 29397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:27.160584 29397 hybrid_clock.cc:648] HybridClock initialized: now 1786091607160567 us; error 43 us; skew 500 ppm
Aug 07 08:33:27 dist-test-slave-6zbq krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:27.167543 29397 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:33:27.168298 29397 webserver.cc:460] Webserver started at http://127.1.157.66:46855/ using document root <none> and password file <none>
I20260807 08:33:27.168747 29397 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:27.168802 29397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:27.168949 29397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:27.170908 29397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "9d06903324de40ada5f8feaa7ae8f42a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "76501e3530a6649273f716d362a886fc"
server_key_iv: "841d46fcb589e404f884ad07b936c0dd"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.171394 29397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "9d06903324de40ada5f8feaa7ae8f42a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "76501e3530a6649273f716d362a886fc"
server_key_iv: "841d46fcb589e404f884ad07b936c0dd"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.174644 29397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:27.176819 29418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.177913 29397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:27.178006 29397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "9d06903324de40ada5f8feaa7ae8f42a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "76501e3530a6649273f716d362a886fc"
server_key_iv: "841d46fcb589e404f884ad07b936c0dd"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.178097 29397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:27.188868 29397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:27.191427 29397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:27.191576 29397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:27.192070 29397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:27.192792 29397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:27.192836 29397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.192878 29397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:27.192907 29397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.200845 29397 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:38401
I20260807 08:33:27.200999 29599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:38401 every 8 connection(s)
I20260807 08:33:27.201666 29397 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:27.209872 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 29397
I20260807 08:33:27.209998 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:27.210258 1653 external_mini_cluster.cc:1428] Setting key 5c7a341f1a8c4eb859dd3cf94882acd6
Aug 07 08:33:27 dist-test-slave-6zbq krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:27.214197 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.203335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:42121 (local address 127.1.157.126:45097)
0807 08:33:27.203465 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.203469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.204189 (+ 720us) server_negotiation.cc:407] Connection header received
0807 08:33:27.205388 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.205391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.205447 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.205521 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:27.206621 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.207148 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.208255 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.208484 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.208573 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.210893 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:27.210897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:27.210900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:27.210924 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:27.212344 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.212812 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.212816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.212817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.212907 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.213171 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.213173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.213175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.213378 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:27.213470 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.213909 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.214036 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:33:27.214893 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.203651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45097 (local address 127.1.157.66:42121)
0807 08:33:27.204056 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.204095 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.205182 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.205685 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.205691 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.206007 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:27.206484 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.206498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.207272 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.207275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.208142 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.208152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.208614 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.208618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.208771 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.209238 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:27.209253 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:27.210744 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:27.212458 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.212463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.212473 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.212724 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.212981 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.212983 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.212985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.213100 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.213458 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:27.213462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:27.213688 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.213844 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.214128 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260807 08:33:27.215977 29600 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45097
I20260807 08:33:27.216218 29600 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:27.216760 29600 heartbeater.cc:511] Master 127.1.157.126:45097 requested a full tablet report, sending...
I20260807 08:33:27.217716 29096 ts_manager.cc:194] Registered new tserver with Master: 9d06903324de40ada5f8feaa7ae8f42a (127.1.157.66:38401)
I20260807 08:33:27.218235 29096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:42121
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:33:27.234746 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:27.346501 29608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:27.346733 29608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:27.346788 29608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:27.350668 29608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:27.350713 29608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:27.350787 29608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:27.355918 29608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:45097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:27.356810 29608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:27.357710 29608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:27.366477 29615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:27.366472 29614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.366626 29608 server_base.cc:1034] running on GCE node
W20260807 08:33:27.366473 29617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.367123 29608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:27.367687 29608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:27.368834 29608 hybrid_clock.cc:648] HybridClock initialized: now 1786091607368808 us; error 40 us; skew 500 ppm
Aug 07 08:33:27 dist-test-slave-6zbq krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:27.371347 29608 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:33:27.372139 29608 webserver.cc:460] Webserver started at http://127.1.157.67:37957/ using document root <none> and password file <none>
I20260807 08:33:27.372632 29608 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:27.372696 29608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:27.372900 29608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:27.374825 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "ad1cce59fe0c4fd69845019d4438202e"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "824baffa43930ccfab956415fd40b6b7"
server_key_iv: "fbea4f52b7de81f72cf630999a8b75ff"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.375344 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "ad1cce59fe0c4fd69845019d4438202e"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "824baffa43930ccfab956415fd40b6b7"
server_key_iv: "fbea4f52b7de81f72cf630999a8b75ff"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.378767 29608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:33:27.381050 29624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.382288 29608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:27.382409 29608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "ad1cce59fe0c4fd69845019d4438202e"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "824baffa43930ccfab956415fd40b6b7"
server_key_iv: "fbea4f52b7de81f72cf630999a8b75ff"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.382493 29608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:27.418303 29608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:27.420815 29608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:27.420981 29608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:27.421514 29608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:27.422276 29608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:27.422323 29608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.422394 29608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:27.422422 29608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.430486 29608 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:35407
I20260807 08:33:27.430536 29802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:35407 every 8 connection(s)
I20260807 08:33:27.431288 29608 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
Aug 07 08:33:27 dist-test-slave-6zbq krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091607, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:27.440671 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 29608
I20260807 08:33:27.440791 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:27.441061 1653 external_mini_cluster.cc:1428] Setting key a86185d069b926e581bf4e3fd76a9c9d
I20260807 08:33:27.443781 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.432909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:57481 (local address 127.1.157.126:45097)
0807 08:33:27.433039 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.433042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.433777 (+ 735us) server_negotiation.cc:407] Connection header received
0807 08:33:27.435029 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.435033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.435090 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.435164 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:27.436287 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.436812 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.437900 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.438125 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.438216 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.440429 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:27.440432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:27.440434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:27.440456 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:27.441849 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.442425 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.442429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.442431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.442513 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.442844 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.442847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.442849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.443019 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:27.443113 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.443564 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.443661 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 08:33:27.444535 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.433223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:45097 (local address 127.1.157.67:57481)
0807 08:33:27.433641 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.433681 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.434829 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.435335 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.435341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.435666 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:27.436128 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.436143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.436933 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.436936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.437775 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.437785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.438260 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.438264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.438458 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.438951 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:27.438967 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:27.440276 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:27.441968 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.441973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.441985 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.442321 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.442597 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:27.442600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:27.442602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:27.442745 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:27.443131 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:27.443136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:27.443353 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.443516 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.443742 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":112,"threads_started":1}
I20260807 08:33:27.445477 29803 heartbeater.cc:347] Connected to a master server at 127.1.157.126:45097
I20260807 08:33:27.445662 29803 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:27.446049 29803 heartbeater.cc:511] Master 127.1.157.126:45097 requested a full tablet report, sending...
I20260807 08:33:27.446940 29096 ts_manager.cc:194] Registered new tserver with Master: ad1cce59fe0c4fd69845019d4438202e (127.1.157.67:35407)
I20260807 08:33:27.447439 29096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:57481
I20260807 08:33:27.454509 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:27.463169 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.456277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.1.157.126:45097)
0807 08:33:27.456389 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.456393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.456544 (+ 151us) server_negotiation.cc:407] Connection header received
0807 08:33:27.456650 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.456653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.456692 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.456753 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:27.457397 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.457882 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.458718 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.458931 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.459012 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.460406 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:27.460410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:27.460413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:27.460437 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:27.461488 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.461959 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.461962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.461964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.462034 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:27.462371 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:27.462374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:27.462376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:27.462564 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:27.462695 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.462950 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.463030 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:33:27.465540 29096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:27.468461 29096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:27.482048 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.476568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.1.157.66:38401)
0807 08:33:27.477736 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.477747 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.477778 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:33:27.477849 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.477855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.478030 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.478177 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:27.478421 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.479325 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.480620 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481405 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.481496 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.481637 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.481811 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.481906 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":115,"threads_started":1}
I20260807 08:33:27.482143 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.475890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:38401 (local address 127.0.0.1:42250)
0807 08:33:27.476207 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.476252 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.476398 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.478171 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.478174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.478194 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:27.478291 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.478297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.479445 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.479448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.480487 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.480498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481521 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.481524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.481631 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.481667 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.481771 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.481906 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":77,"thread_start_us":91,"threads_started":1}
I20260807 08:33:27.482275 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.475779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:36995 (local address 127.0.0.1:40516)
0807 08:33:27.476206 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.476250 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.476368 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.477451 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.477457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.477481 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:27.477599 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.477606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.479030 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.479033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.480497 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.480507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481727 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.481730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.481828 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.481851 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.481902 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.481967 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":61,"thread_start_us":59,"threads_started":1}
I20260807 08:33:27.482290 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.476417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35407 (local address 127.0.0.1:60358)
0807 08:33:27.476747 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.476761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.476840 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.477700 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.477702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.477717 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:27.477778 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.477781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.478862 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.478866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.480622 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.480630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481929 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.481934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.482019 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.482039 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.482100 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.482161 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":70,"thread_start_us":85,"threads_started":1}
I20260807 08:33:27.482543 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.475893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.1.157.65:36995)
0807 08:33:27.477031 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.477044 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.477061 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:33:27.477122 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.477126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.477279 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.477432 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:27.477716 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.478886 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.480691 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481592 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.482117 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.482248 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.482297 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.482413 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":63,"threads_started":1}
I20260807 08:33:27.482666 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.476687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.1.157.67:35407)
0807 08:33:27.476982 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.476993 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.477014 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:33:27.477086 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.477092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.477290 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.477431 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:27.477876 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.478742 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.480737 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.481592 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.481692 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.481822 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.482460 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.482547 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260807 08:33:27.485157 29698 tablet_service.cc:1467] Processing CreateTablet for tablet 678f06f9701e4597b7caf473fb53e270 (DEFAULT_TABLE table=test-table [id=eb60fc0f33b5422ea9fb4cc744255c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:27.485240 29494 tablet_service.cc:1467] Processing CreateTablet for tablet 678f06f9701e4597b7caf473fb53e270 (DEFAULT_TABLE table=test-table [id=eb60fc0f33b5422ea9fb4cc744255c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:27.485237 29294 tablet_service.cc:1467] Processing CreateTablet for tablet 678f06f9701e4597b7caf473fb53e270 (DEFAULT_TABLE table=test-table [id=eb60fc0f33b5422ea9fb4cc744255c02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:27.486160 29698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678f06f9701e4597b7caf473fb53e270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:27.486248 29494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678f06f9701e4597b7caf473fb53e270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:27.486248 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678f06f9701e4597b7caf473fb53e270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:27.491995 29823 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Bootstrap starting.
I20260807 08:33:27.493736 29825 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: Bootstrap starting.
I20260807 08:33:27.494314 29823 tablet_bootstrap.cc:654] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:27.494690 29828 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Bootstrap starting.
I20260807 08:33:27.495422 29825 tablet_bootstrap.cc:654] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:27.495563 29823 log.cc:824] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:27.496466 29825 log.cc:824] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:27.496569 29828 tablet_bootstrap.cc:654] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:27.497645 29823 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: No bootstrap required, opened a new log
I20260807 08:33:27.497680 29828 log.cc:824] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:27.497851 29823 ts_tablet_manager.cc:1397] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:33:27.497987 29825 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: No bootstrap required, opened a new log
I20260807 08:33:27.498162 29825 ts_tablet_manager.cc:1397] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 08:33:27.499548 29828 tablet_bootstrap.cc:492] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: No bootstrap required, opened a new log
I20260807 08:33:27.499722 29828 ts_tablet_manager.cc:1397] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:27.500949 29823 raft_consensus.cc:348] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.501178 29823 raft_consensus.cc:374] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:27.501250 29823 raft_consensus.cc:729] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1cce59fe0c4fd69845019d4438202e, State: Initialized, Role: FOLLOWER
I20260807 08:33:27.501658 29823 consensus_queue.cc:260] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.501791 29825 raft_consensus.cc:348] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.502092 29825 raft_consensus.cc:374] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:27.502184 29825 raft_consensus.cc:729] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b363ddafd4b74c26ac576b4e7259f7f2, State: Initialized, Role: FOLLOWER
I20260807 08:33:27.502250 29803 heartbeater.cc:503] Master 127.1.157.126:45097 was elected leader, sending a full tablet report...
I20260807 08:33:27.502633 29823 ts_tablet_manager.cc:1428] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 08:33:27.502712 29825 consensus_queue.cc:260] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.503440 29391 heartbeater.cc:503] Master 127.1.157.126:45097 was elected leader, sending a full tablet report...
I20260807 08:33:27.503362 29828 raft_consensus.cc:348] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.503618 29828 raft_consensus.cc:374] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:27.503675 29828 raft_consensus.cc:729] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d06903324de40ada5f8feaa7ae8f42a, State: Initialized, Role: FOLLOWER
I20260807 08:33:27.504076 29828 consensus_queue.cc:260] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.505004 29828 ts_tablet_manager.cc:1428] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 08:33:27.505086 29600 heartbeater.cc:503] Master 127.1.157.126:45097 was elected leader, sending a full tablet report...
I20260807 08:33:27.506621 29825 ts_tablet_manager.cc:1428] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260807 08:33:27.518249 29834 raft_consensus.cc:482] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:27.518410 29834 raft_consensus.cc:504] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.519251 29834 leader_election.cc:288] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b363ddafd4b74c26ac576b4e7259f7f2 (127.1.157.65:36995), ad1cce59fe0c4fd69845019d4438202e (127.1.157.67:35407)
I20260807 08:33:27.523643 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.519747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:42599 (local address 127.1.157.67:35407)
0807 08:33:27.519854 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.519858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.520022 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:33:27.520106 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.520108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.520141 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.520198 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:27.520426 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.521109 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.522474 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.523079 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.523197 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.523248 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.523464 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.523525 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:33:27.523643 29817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.519497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:45333 (local address 127.1.157.65:36995)
0807 08:33:27.519602 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.519605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.519619 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:33:27.519849 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.519852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.519898 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.519968 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:27.520239 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.520985 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.522476 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.523079 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.523197 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.523247 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.523464 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.523530 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:33:27.523713 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.519424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:36995 (local address 127.1.157.66:45333)
0807 08:33:27.519534 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.519549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.519626 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.520003 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.520006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.520025 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:27.520139 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.520145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.521099 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.521101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.522331 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.522367 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.523183 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.523186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.523326 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.523364 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.523442 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.523541 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":17}
I20260807 08:33:27.523722 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.519674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:35407 (local address 127.1.157.66:42599)
0807 08:33:27.519939 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.519951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:27.520036 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:27.520222 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:27.520225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:27.520243 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:27.520340 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.520345 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.521195 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.521198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.522366 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:27.522373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.523183 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.523186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:27.523330 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.523377 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 08:33:27.523435 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 08:33:27.523541 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":70,"thread_start_us":85,"threads_started":1}
I20260807 08:33:27.524324 29730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678f06f9701e4597b7caf473fb53e270" candidate_uuid: "9d06903324de40ada5f8feaa7ae8f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1cce59fe0c4fd69845019d4438202e" is_pre_election: true
I20260807 08:33:27.524324 29328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678f06f9701e4597b7caf473fb53e270" candidate_uuid: "9d06903324de40ada5f8feaa7ae8f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" is_pre_election: true
I20260807 08:33:27.524595 29730 raft_consensus.cc:2454] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d06903324de40ada5f8feaa7ae8f42a in term 0.
I20260807 08:33:27.524595 29328 raft_consensus.cc:2454] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d06903324de40ada5f8feaa7ae8f42a in term 0.
I20260807 08:33:27.525009 29420 leader_election.cc:302] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d06903324de40ada5f8feaa7ae8f42a, b363ddafd4b74c26ac576b4e7259f7f2; no voters:
I20260807 08:33:27.525238 29834 raft_consensus.cc:2781] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:27.525317 29834 raft_consensus.cc:482] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:27.525347 29834 raft_consensus.cc:3037] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:27.526248 29834 raft_consensus.cc:504] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.526564 29834 leader_election.cc:288] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [CANDIDATE]: Term 1 election: Requested vote from peers b363ddafd4b74c26ac576b4e7259f7f2 (127.1.157.65:36995), ad1cce59fe0c4fd69845019d4438202e (127.1.157.67:35407)
I20260807 08:33:27.526886 29328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678f06f9701e4597b7caf473fb53e270" candidate_uuid: "9d06903324de40ada5f8feaa7ae8f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b363ddafd4b74c26ac576b4e7259f7f2"
I20260807 08:33:27.526937 29730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678f06f9701e4597b7caf473fb53e270" candidate_uuid: "9d06903324de40ada5f8feaa7ae8f42a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1cce59fe0c4fd69845019d4438202e"
I20260807 08:33:27.527004 29328 raft_consensus.cc:3037] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:27.527031 29730 raft_consensus.cc:3037] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:27.527935 29730 raft_consensus.cc:2454] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d06903324de40ada5f8feaa7ae8f42a in term 1.
I20260807 08:33:27.527938 29328 raft_consensus.cc:2454] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d06903324de40ada5f8feaa7ae8f42a in term 1.
I20260807 08:33:27.528198 29420 leader_election.cc:302] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d06903324de40ada5f8feaa7ae8f42a, b363ddafd4b74c26ac576b4e7259f7f2; no voters:
I20260807 08:33:27.528353 29834 raft_consensus.cc:2781] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:27.528556 29834 raft_consensus.cc:686] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 1 LEADER]: Becoming Leader. State: Replica: 9d06903324de40ada5f8feaa7ae8f42a, State: Running, Role: LEADER
I20260807 08:33:27.528843 29834 consensus_queue.cc:237] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } }
I20260807 08:33:27.531631 29096 catalog_manager.cc:5495] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d06903324de40ada5f8feaa7ae8f42a (127.1.157.66). New cstate: current_term: 1 leader_uuid: "9d06903324de40ada5f8feaa7ae8f42a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d06903324de40ada5f8feaa7ae8f42a" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 38401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 } health_report { overall_health: UNKNOWN } } }
I20260807 08:33:27.544716 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.541345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.1.157.66:38401)
0807 08:33:27.541456 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.541460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.541473 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:33:27.541622 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.541625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.541672 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.541754 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:27.541986 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.542491 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.543334 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.543543 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.543629 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.543952 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:27.544304 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:27.544430 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.544554 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.544589 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:33:27.549887 29730 raft_consensus.cc:1261] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 1 FOLLOWER]: Refusing update from remote peer 9d06903324de40ada5f8feaa7ae8f42a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:27.549887 29328 raft_consensus.cc:1261] T 678f06f9701e4597b7caf473fb53e270 P b363ddafd4b74c26ac576b4e7259f7f2 [term 1 FOLLOWER]: Refusing update from remote peer 9d06903324de40ada5f8feaa7ae8f42a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:27.550537 29834 consensus_queue.cc:1035] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b363ddafd4b74c26ac576b4e7259f7f2" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 36995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:27.550724 29839 consensus_queue.cc:1035] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad1cce59fe0c4fd69845019d4438202e" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 35407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:27.558256 29850 mvcc.cc:205] Tried to move back new op lower bound from 7315831224517672960 to 7315831224440508416. Current Snapshot: MvccSnapshot[applied={T|T < 7315831224517672960}]
I20260807 08:33:27.558929 29849 mvcc.cc:205] Tried to move back new op lower bound from 7315831224517672960 to 7315831224440508416. Current Snapshot: MvccSnapshot[applied={T|T < 7315831224517672960}]
I20260807 08:33:27.565961 29085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:27.566129 29085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:27.568382 29085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9db20bc579a243278331281f7f8cb09a: Sending DeleteTablet for 3 replicas of tablet 678f06f9701e4597b7caf473fb53e270
I20260807 08:33:27.569073 29698 tablet_service.cc:1514] Processing DeleteTablet for tablet 678f06f9701e4597b7caf473fb53e270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:60358
I20260807 08:33:27.569118 29494 tablet_service.cc:1514] Processing DeleteTablet for tablet 678f06f9701e4597b7caf473fb53e270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:42250
I20260807 08:33:27.569540 29294 tablet_service.cc:1514] Processing DeleteTablet for tablet 678f06f9701e4597b7caf473fb53e270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:27 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:40516
I20260807 08:33:27.569643 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 29193
I20260807 08:33:27.572094 29867 tablet_replica.cc:330] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: stopping tablet replica
I20260807 08:33:27.572403 29867 raft_consensus.cc:2229] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:33:27.572772 29867 raft_consensus.cc:2258] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:27.573132 29866 tablet_replica.cc:330] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: stopping tablet replica
I20260807 08:33:27.576145 29867 ts_tablet_manager.cc:1905] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:27.576455 29866 raft_consensus.cc:2229] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:27.576763 29866 raft_consensus.cc:2258] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:27.579450 29867 ts_tablet_manager.cc:1918] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:27.579854 29867 log.cc:1196] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/678f06f9701e4597b7caf473fb53e270
I20260807 08:33:27.580272 29867 ts_tablet_manager.cc:1939] T 678f06f9701e4597b7caf473fb53e270 P 9d06903324de40ada5f8feaa7ae8f42a: Deleting consensus metadata
I20260807 08:33:27.581714 29866 ts_tablet_manager.cc:1905] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:27.582329 29072 catalog_manager.cc:4841] TS 9d06903324de40ada5f8feaa7ae8f42a (127.1.157.66:38401): tablet 678f06f9701e4597b7caf473fb53e270 (table test-table [id=eb60fc0f33b5422ea9fb4cc744255c02]) successfully deleted
I20260807 08:33:27.584438 29866 ts_tablet_manager.cc:1918] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:27.584535 29866 log.cc:1196] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/678f06f9701e4597b7caf473fb53e270
I20260807 08:33:27.584858 29866 ts_tablet_manager.cc:1939] T 678f06f9701e4597b7caf473fb53e270 P ad1cce59fe0c4fd69845019d4438202e: Deleting consensus metadata
I20260807 08:33:27.585968 29074 catalog_manager.cc:4841] TS ad1cce59fe0c4fd69845019d4438202e (127.1.157.67:35407): tablet 678f06f9701e4597b7caf473fb53e270 (table test-table [id=eb60fc0f33b5422ea9fb4cc744255c02]) successfully deleted
W20260807 08:33:27.588295 29074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:36995 (error 108)
I20260807 08:33:27.588878 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 29397
I20260807 08:33:27.589265 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.589003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:36995 (local address 127.0.0.1:40520)
0807 08:33:27.589124 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:27.589189 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:36995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:33:27.595716 29074 catalog_manager.cc:4584] TS b363ddafd4b74c26ac576b4e7259f7f2 (127.1.157.65:36995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 678f06f9701e4597b7caf473fb53e270: Network error: Client connection negotiation failed: client connection to 127.1.157.65:36995: connect: Connection refused (error 111)
I20260807 08:33:27.601189 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 29608
I20260807 08:33:27.613744 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 29053
2026-08-07T08:33:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2231 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T08:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:27Z Disabled control of system clock
I20260807 08:33:27.648082 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:41403
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:27.753069 29874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:27.753293 29874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:27.753327 29874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:27.757149 29874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:27.757196 29874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:27.757210 29874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:27.757226 29874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:27.757256 29874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:27.761759 29874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:41403
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:27.762707 29874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:27.763912 29874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:27.771420 29881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.771489 29874 server_base.cc:1034] running on GCE node
W20260807 08:33:27.771500 29883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:27.771430 29885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.772184 29874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:27.773147 29874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:27.774295 29874 hybrid_clock.cc:648] HybridClock initialized: now 1786091607774270 us; error 37 us; skew 500 ppm
I20260807 08:33:27.775874 29874 webserver.cc:460] Webserver started at http://127.1.157.126:40429/ using document root <none> and password file <none>
I20260807 08:33:27.776332 29874 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:27.776384 29874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:27.776536 29874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:27.778287 29874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "2086a312c8e74d2dbcc0dbd1eadc656a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "496d647bad6a40cd7978207c29bfd6d6"
server_key_iv: "ef69643499bcf252674f1e689e507e1d"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.778810 29874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "2086a312c8e74d2dbcc0dbd1eadc656a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "496d647bad6a40cd7978207c29bfd6d6"
server_key_iv: "ef69643499bcf252674f1e689e507e1d"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.781921 29874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:33:27.784055 29896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:27.785100 29874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:33:27.785192 29874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "2086a312c8e74d2dbcc0dbd1eadc656a"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "496d647bad6a40cd7978207c29bfd6d6"
server_key_iv: "ef69643499bcf252674f1e689e507e1d"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.785266 29874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:27.800053 29874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:27.800545 29874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:27.800678 29874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:27.807653 29874 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:41403
I20260807 08:33:27.807655 29968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:41403 every 8 connection(s)
I20260807 08:33:27.808492 29874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:27.811326 29969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:27.813120 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 29874
I20260807 08:33:27.813215 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:27.813464 1653 external_mini_cluster.cc:1428] Setting key 63474e5187406ae753520a560395fcfc
I20260807 08:33:27.817149 29969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: Bootstrap starting.
I20260807 08:33:27.819554 29969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:27.820384 29969 log.cc:824] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:27.822479 29969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: No bootstrap required, opened a new log
I20260807 08:33:27.823143 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:27.814757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.1.157.126:41403)
0807 08:33:27.815383 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:27.815402 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:27.815443 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:33:27.816217 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:27.816240 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:27.816691 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:27.817033 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:27.817507 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.818925 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.819803 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:27.820461 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:27.820610 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:27.821309 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:27.821314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:27.821329 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:27.821758 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:27.821827 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:27.821852 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:27.821948 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:27.822032 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 08:33:27.822281 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":97,"threads_started":1}
I20260807 08:33:27.826231 29969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } }
I20260807 08:33:27.826473 29969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:27.826519 29969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2086a312c8e74d2dbcc0dbd1eadc656a, State: Initialized, Role: FOLLOWER
I20260807 08:33:27.826874 29969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } }
I20260807 08:33:27.826990 29969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:27.827034 29969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:27.827100 29969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:27.827908 29969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } }
I20260807 08:33:27.828177 29969 leader_election.cc:302] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2086a312c8e74d2dbcc0dbd1eadc656a; no voters:
I20260807 08:33:27.828397 29969 leader_election.cc:288] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:27.828501 29975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:27.828720 29975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [term 1 LEADER]: Becoming Leader. State: Replica: 2086a312c8e74d2dbcc0dbd1eadc656a, State: Running, Role: LEADER
I20260807 08:33:27.829006 29975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } }
I20260807 08:33:27.829478 29969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:27.830520 29976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } } }
I20260807 08:33:27.830647 29976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:27.831023 29986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:27.831177 29977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2086a312c8e74d2dbcc0dbd1eadc656a. Latest consensus state: current_term: 1 leader_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2086a312c8e74d2dbcc0dbd1eadc656a" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 41403 } } }
I20260807 08:33:27.831283 29977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:27.833787 29986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:27.838505 29986 catalog_manager.cc:1347] Generated new cluster ID: d3ae1607cb8740bfaba4f9b0b6a78d92
I20260807 08:33:27.838567 29986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:27.847550 29986 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:27.848615 29986 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:27.858747 29986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: Generated new TSK 0
I20260807 08:33:27.859403 29986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:27.864168 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--builtin_ntp_servers=127.1.157.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:27.971012 30004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:27.971221 30004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:27.971256 30004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:27.975066 30004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:27.975112 30004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:27.975162 30004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:27.979614 30004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:27.980453 30004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:27.981642 30004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:27.990279 30023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:27.990293 30020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.990470 30004 server_base.cc:1034] running on GCE node
W20260807 08:33:27.990293 30018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:27.990953 30004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:27.991510 30004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:27.992666 30004 hybrid_clock.cc:648] HybridClock initialized: now 1786091607992638 us; error 41 us; skew 500 ppm
I20260807 08:33:27.994282 30004 webserver.cc:460] Webserver started at http://127.1.157.65:44161/ using document root <none> and password file <none>
I20260807 08:33:27.994808 30004 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:27.994874 30004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:27.995042 30004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:27.996861 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "7243b68e40d24651b855d65120d7769d"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "b94fa378d722986d1aa05849251141ff"
server_key_iv: "a2e60caaad426547293bb0c932eb9206"
server_key_version: "encryptionkey@0"
I20260807 08:33:27.997372 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "7243b68e40d24651b855d65120d7769d"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "b94fa378d722986d1aa05849251141ff"
server_key_iv: "a2e60caaad426547293bb0c932eb9206"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.000660 30004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:33:28.002810 30035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.003851 30004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:28.003955 30004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "7243b68e40d24651b855d65120d7769d"
format_stamp: "Formatted at 2026-08-07 08:33:27 on dist-test-slave-6zbq"
server_key: "b94fa378d722986d1aa05849251141ff"
server_key_iv: "a2e60caaad426547293bb0c932eb9206"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.004041 30004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:28.036229 30004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:28.036772 30004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:28.036919 30004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:28.037420 30004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:28.038239 30004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:28.038300 30004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.038368 30004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:28.038400 30004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.046514 30004 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:42689
I20260807 08:33:28.046543 30211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:42689 every 8 connection(s)
I20260807 08:33:28.047300 30004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:28.050555 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 30004
I20260807 08:33:28.050676 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:28.050987 1653 external_mini_cluster.cc:1428] Setting key 93658952fd08b247308a72630f3b6bd5
I20260807 08:33:28.053467 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--builtin_ntp_servers=127.1.157.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:28.056249 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.048973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:41071 (local address 127.1.157.126:41403)
0807 08:33:28.049097 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.049101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.050029 (+ 928us) server_negotiation.cc:407] Connection header received
0807 08:33:28.051655 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.051658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.051715 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.051798 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:28.052621 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.053295 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.054476 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.054776 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.054876 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.055816 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:28.055821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:28.055823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:28.055837 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:28.055859 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:28.055869 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:28.055943 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.056049 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.056110 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 08:33:28.056939 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.049309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:41403 (local address 127.1.157.65:41071)
0807 08:33:28.049897 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.049934 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.051463 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.051945 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.051952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.052030 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:28.052488 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.052501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.053423 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.053426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.054323 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.054333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.054902 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.054905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.055037 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.055570 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:28.055572 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:28.055608 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:28.055666 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:28.055943 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:28.055947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:28.055951 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.056031 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.056227 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
I20260807 08:33:28.058070 30212 heartbeater.cc:347] Connected to a master server at 127.1.157.126:41403
I20260807 08:33:28.058306 30212 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:28.058902 30212 heartbeater.cc:511] Master 127.1.157.126:41403 requested a full tablet report, sending...
I20260807 08:33:28.060354 29909 ts_manager.cc:194] Registered new tserver with Master: 7243b68e40d24651b855d65120d7769d (127.1.157.65:42689)
I20260807 08:33:28.061439 29909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.65:41071
W20260807 08:33:28.160575 30217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:28.160800 30217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:28.160835 30217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:28.164695 30217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:28.164736 30217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:28.164788 30217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:28.169343 30217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:28.170222 30217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:28.171465 30217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:28.180087 30222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:28.180094 30224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:28.180166 30217 server_base.cc:1034] running on GCE node
W20260807 08:33:28.180100 30229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:28.180657 30217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:28.181216 30217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:28.182387 30217 hybrid_clock.cc:648] HybridClock initialized: now 1786091608182365 us; error 39 us; skew 500 ppm
I20260807 08:33:28.183990 30217 webserver.cc:460] Webserver started at http://127.1.157.66:41311/ using document root <none> and password file <none>
I20260807 08:33:28.184499 30217 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:28.184563 30217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:28.184738 30217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:28.186637 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "f6458a1406e64303a885f09e9d64920f"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "ccf2561caad291bc08dad0364d51ddf9"
server_key_iv: "1e892d4a85fbe211ed99372e74314162"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.187143 30217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "f6458a1406e64303a885f09e9d64920f"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "ccf2561caad291bc08dad0364d51ddf9"
server_key_iv: "1e892d4a85fbe211ed99372e74314162"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.190629 30217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 08:33:28.192770 30239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.193826 30217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:28.193928 30217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "f6458a1406e64303a885f09e9d64920f"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "ccf2561caad291bc08dad0364d51ddf9"
server_key_iv: "1e892d4a85fbe211ed99372e74314162"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.194015 30217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:28.205973 30217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:28.206597 30217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:28.206750 30217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:28.207247 30217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:28.208117 30217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:28.208163 30217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.208211 30217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:28.208240 30217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.215976 30217 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:44783
I20260807 08:33:28.215999 30418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:44783 every 8 connection(s)
I20260807 08:33:28.216744 30217 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:28.219972 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 30217
I20260807 08:33:28.220082 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:28.220336 1653 external_mini_cluster.cc:1428] Setting key e6d87c3680f8bb9622f0fa1c677bf7d3
I20260807 08:33:28.222200 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--builtin_ntp_servers=127.1.157.84:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:28.225540 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.218319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:45799 (local address 127.1.157.126:41403)
0807 08:33:28.218472 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.218477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.219440 (+ 963us) server_negotiation.cc:407] Connection header received
0807 08:33:28.220784 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.220787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.220843 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.220907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:28.221805 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.222565 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.223640 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.223932 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.224012 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.225144 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:28.225147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:28.225149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:28.225159 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:28.225179 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:28.225187 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:28.225235 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.225382 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.225431 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:33:28.226536 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.218710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:41403 (local address 127.1.157.66:45799)
0807 08:33:28.219305 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.219341 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.220570 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.221073 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.221093 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.221171 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:28.221670 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.221684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.222678 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.222681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.223507 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.223516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.224088 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.224093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.224287 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.224858 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:28.224860 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:28.224914 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:28.224990 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:28.225255 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:28.225264 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:28.225270 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.225384 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.225644 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
I20260807 08:33:28.227581 30419 heartbeater.cc:347] Connected to a master server at 127.1.157.126:41403
I20260807 08:33:28.227785 30419 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:28.228219 30419 heartbeater.cc:511] Master 127.1.157.126:41403 requested a full tablet report, sending...
I20260807 08:33:28.229218 29909 ts_manager.cc:194] Registered new tserver with Master: f6458a1406e64303a885f09e9d64920f (127.1.157.66:44783)
I20260807 08:33:28.229943 29909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.66:45799
W20260807 08:33:28.328943 30422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:28.329170 30422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:28.329205 30422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:28.333186 30422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:28.333233 30422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:28.333283 30422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:28.337779 30422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:28.338657 30422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:28.339848 30422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:28.348474 30433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:28.348510 30434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:28.348618 30422 server_base.cc:1034] running on GCE node
W20260807 08:33:28.348479 30437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:28.349138 30422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:28.349686 30422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:28.350836 30422 hybrid_clock.cc:648] HybridClock initialized: now 1786091608350820 us; error 26 us; skew 500 ppm
I20260807 08:33:28.352458 30422 webserver.cc:460] Webserver started at http://127.1.157.67:42269/ using document root <none> and password file <none>
I20260807 08:33:28.352962 30422 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:28.353027 30422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:28.353214 30422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:28.355118 30422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "80e93d70a3734ec698623e810738cc00"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "432709096cb7953ad8a790841c53a7d7"
server_key_iv: "4f36b4c706a06f566f991d09846c6460"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.355638 30422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "80e93d70a3734ec698623e810738cc00"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "432709096cb7953ad8a790841c53a7d7"
server_key_iv: "4f36b4c706a06f566f991d09846c6460"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.359066 30422 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:28.361245 30444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.362317 30422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:28.362440 30422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "80e93d70a3734ec698623e810738cc00"
format_stamp: "Formatted at 2026-08-07 08:33:28 on dist-test-slave-6zbq"
server_key: "432709096cb7953ad8a790841c53a7d7"
server_key_iv: "4f36b4c706a06f566f991d09846c6460"
server_key_version: "encryptionkey@0"
I20260807 08:33:28.362524 30422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:28.395931 30422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:28.396500 30422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:28.396653 30422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:28.397166 30422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:28.398018 30422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:28.398064 30422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.398120 30422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:28.398146 30422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:28.405905 30422 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:36431
I20260807 08:33:28.405933 30635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:36431 every 8 connection(s)
I20260807 08:33:28.406721 30422 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:33:28.409271 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 30422
I20260807 08:33:28.409373 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:28.409636 1653 external_mini_cluster.cc:1428] Setting key 690d2323469dbf10f28dbaae36798dfd
I20260807 08:33:28.415337 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.408388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:45329 (local address 127.1.157.126:41403)
0807 08:33:28.408505 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.408508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.409418 (+ 910us) server_negotiation.cc:407] Connection header received
0807 08:33:28.410966 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.410969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.411026 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.411098 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:28.412005 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.412540 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.413625 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.413849 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.413935 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.414893 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:28.414896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:28.414899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:28.414912 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:28.414930 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:28.414938 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:28.414995 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.415133 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.415192 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:33:28.416049 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.408708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:41403 (local address 127.1.157.67:45329)
0807 08:33:28.409288 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.409325 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.410758 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.411265 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.411273 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.411352 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:28.411869 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.411883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.412666 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.412670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.413504 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.413514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.413982 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.413986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.414134 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.414650 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:28.414652 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:28.414688 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:28.414745 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:28.415005 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:28.415014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:28.415020 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.415121 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.415369 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":113,"threads_started":1}
I20260807 08:33:28.417008 30636 heartbeater.cc:347] Connected to a master server at 127.1.157.126:41403
I20260807 08:33:28.417217 30636 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:28.417649 30636 heartbeater.cc:511] Master 127.1.157.126:41403 requested a full tablet report, sending...
I20260807 08:33:28.418535 29909 ts_manager.cc:194] Registered new tserver with Master: 80e93d70a3734ec698623e810738cc00 (127.1.157.67:36431)
I20260807 08:33:28.419049 29909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.67:45329
I20260807 08:33:28.422930 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:28.428256 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.424738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.1.157.126:41403)
0807 08:33:28.424852 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.424855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.425027 (+ 172us) server_negotiation.cc:407] Connection header received
0807 08:33:28.425128 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.425130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.425168 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.425225 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:28.425545 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.426100 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.426945 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.427158 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.427229 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.427896 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:28.427899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:28.427901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:28.427912 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:28.427929 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:28.427937 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:28.428026 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.428107 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.428140 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:33:28.430801 29909 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:28.433650 29909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:28.446578 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.441365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36431 (local address 127.0.0.1:50596)
0807 08:33:28.441840 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.441882 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.442007 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.442756 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.442762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.442782 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:28.442912 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.442920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.444027 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.444030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.445042 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.445053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.446056 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.446059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.446172 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.446211 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.446310 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.446440 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":66,"threads_started":1}
I20260807 08:33:28.446593 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.441498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.1.157.67:36431)
0807 08:33:28.441910 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.441922 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.441951 (+ 29us) server_negotiation.cc:407] Connection header received
0807 08:33:28.442206 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.442212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.442449 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.442590 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:28.443036 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.443932 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.445207 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.445959 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.446047 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.446239 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.446365 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.446455 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":118,"threads_started":1}
I20260807 08:33:28.447139 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.441500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42689 (local address 127.0.0.1:57606)
0807 08:33:28.442134 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.442146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.442226 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.443131 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.443134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.443161 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:28.443241 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.443247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.444619 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.444622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.445786 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.445799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.446776 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.446780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.446866 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.446891 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.446951 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.447018 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":45,"thread_start_us":106,"threads_started":1}
I20260807 08:33:28.447227 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.442104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.1.157.65:42689)
0807 08:33:28.442705 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.442717 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.442741 (+ 24us) server_negotiation.cc:407] Connection header received
0807 08:33:28.442829 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.442834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.443007 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.443153 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:28.443358 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.444501 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.445898 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.446666 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.446777 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.446942 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.446986 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.447066 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":105,"threads_started":1}
I20260807 08:33:28.447849 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.442017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:44783 (local address 127.0.0.1:45766)
0807 08:33:28.442308 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.442322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.442430 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.443410 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.443412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.443429 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:28.443496 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.443501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.444730 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.444733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.445707 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.445718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.447494 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.447498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.447583 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.447606 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.447653 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.447714 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":29,"thread_start_us":80,"threads_started":1}
I20260807 08:33:28.448571 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.442595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.1.157.66:44783)
0807 08:33:28.442968 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.442978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.442999 (+ 21us) server_negotiation.cc:407] Connection header received
0807 08:33:28.443071 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.443076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.443236 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.443430 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:28.443566 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.444634 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.445858 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.447152 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.448106 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.448278 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.448327 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.448417 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":92,"threads_started":1}
I20260807 08:33:28.449330 30524 tablet_service.cc:1467] Processing CreateTablet for tablet bc8e2df352f74e67995d3998bd3d6217 (DEFAULT_TABLE table=test-table [id=6b9c757e8e43454abb1b64f408f8305c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:28.449625 30115 tablet_service.cc:1467] Processing CreateTablet for tablet bc8e2df352f74e67995d3998bd3d6217 (DEFAULT_TABLE table=test-table [id=6b9c757e8e43454abb1b64f408f8305c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:28.450241 30524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8e2df352f74e67995d3998bd3d6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:28.450624 30115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8e2df352f74e67995d3998bd3d6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:28.452378 30329 tablet_service.cc:1467] Processing CreateTablet for tablet bc8e2df352f74e67995d3998bd3d6217 (DEFAULT_TABLE table=test-table [id=6b9c757e8e43454abb1b64f408f8305c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:28.453331 30329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc8e2df352f74e67995d3998bd3d6217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:28.456037 30655 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Bootstrap starting.
I20260807 08:33:28.457998 30656 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Bootstrap starting.
I20260807 08:33:28.458787 30655 tablet_bootstrap.cc:654] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:28.459450 30655 log.cc:824] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:28.460067 30656 tablet_bootstrap.cc:654] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:28.460816 30656 log.cc:824] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:28.461229 30655 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: No bootstrap required, opened a new log
I20260807 08:33:28.461422 30655 ts_tablet_manager.cc:1397] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 08:33:28.462632 30656 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: No bootstrap required, opened a new log
I20260807 08:33:28.462811 30656 ts_tablet_manager.cc:1397] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:28.464699 30655 raft_consensus.cc:348] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.464892 30655 raft_consensus.cc:374] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:28.464926 30655 raft_consensus.cc:729] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e93d70a3734ec698623e810738cc00, State: Initialized, Role: FOLLOWER
I20260807 08:33:28.465298 30655 consensus_queue.cc:260] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.465930 30636 heartbeater.cc:503] Master 127.1.157.126:41403 was elected leader, sending a full tablet report...
I20260807 08:33:28.465828 30656 raft_consensus.cc:348] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.466064 30656 raft_consensus.cc:374] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:28.466115 30656 raft_consensus.cc:729] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7243b68e40d24651b855d65120d7769d, State: Initialized, Role: FOLLOWER
I20260807 08:33:28.466144 30655 ts_tablet_manager.cc:1428] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:28.466491 30660 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Bootstrap starting.
I20260807 08:33:28.466557 30656 consensus_queue.cc:260] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.467460 30656 ts_tablet_manager.cc:1428] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 08:33:28.467753 30212 heartbeater.cc:503] Master 127.1.157.126:41403 was elected leader, sending a full tablet report...
I20260807 08:33:28.468248 30660 tablet_bootstrap.cc:654] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:28.468966 30660 log.cc:824] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:28.470614 30660 tablet_bootstrap.cc:492] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: No bootstrap required, opened a new log
I20260807 08:33:28.470774 30660 ts_tablet_manager.cc:1397] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260807 08:33:28.473839 30660 raft_consensus.cc:348] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.474017 30660 raft_consensus.cc:374] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:28.474072 30660 raft_consensus.cc:729] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6458a1406e64303a885f09e9d64920f, State: Initialized, Role: FOLLOWER
I20260807 08:33:28.474433 30660 consensus_queue.cc:260] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.475008 30419 heartbeater.cc:503] Master 127.1.157.126:41403 was elected leader, sending a full tablet report...
I20260807 08:33:28.475237 30660 ts_tablet_manager.cc:1428] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260807 08:33:28.548354 30213 tablet.cc:2369] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:28.581835 30673 raft_consensus.cc:482] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:28.582005 30673 raft_consensus.cc:504] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.583067 30673 leader_election.cc:288] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80e93d70a3734ec698623e810738cc00 (127.1.157.67:36431), 7243b68e40d24651b855d65120d7769d (127.1.157.65:42689)
I20260807 08:33:28.586695 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.583234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:36431 (local address 127.1.157.66:33949)
0807 08:33:28.583359 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.583374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.583455 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.583770 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.583774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.583818 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:28.583936 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.583943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.584715 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.584719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.585621 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.585631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.586298 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.586302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.586422 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.586449 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.586512 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.586581 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":32,"spinlock_wait_cycles":3328}
I20260807 08:33:28.586740 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.583423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:33949 (local address 127.1.157.67:36431)
0807 08:33:28.583550 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.583554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.583568 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:33:28.583619 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.583622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.583667 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.583749 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:28.584048 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.584617 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.585735 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.586190 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.586287 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.586325 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.586549 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.586619 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 08:33:28.586887 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.583390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:57083 (local address 127.1.157.65:42689)
0807 08:33:28.583505 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.583509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.583760 (+ 251us) server_negotiation.cc:407] Connection header received
0807 08:33:28.583915 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.583918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.583964 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.584034 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:28.584274 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.584867 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.585961 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.586398 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.586506 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.586546 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.586715 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.586784 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:33:28.586898 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.583316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42689 (local address 127.1.157.66:57083)
0807 08:33:28.583684 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.583697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:28.583823 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:28.584045 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:28.584047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:28.584062 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:28.584182 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.584188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.584976 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.584979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.585839 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:28.585850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.586498 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.586502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:28.586599 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.586621 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 08:33:28.586692 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 08:33:28.586750 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":37,"spinlock_wait_cycles":38016,"thread_start_us":112,"threads_started":1}
I20260807 08:33:28.587247 30554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8e2df352f74e67995d3998bd3d6217" candidate_uuid: "f6458a1406e64303a885f09e9d64920f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e93d70a3734ec698623e810738cc00" is_pre_election: true
I20260807 08:33:28.587524 30554 raft_consensus.cc:2454] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6458a1406e64303a885f09e9d64920f in term 0.
I20260807 08:33:28.587535 30143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8e2df352f74e67995d3998bd3d6217" candidate_uuid: "f6458a1406e64303a885f09e9d64920f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7243b68e40d24651b855d65120d7769d" is_pre_election: true
I20260807 08:33:28.587767 30143 raft_consensus.cc:2454] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6458a1406e64303a885f09e9d64920f in term 0.
I20260807 08:33:28.587989 30242 leader_election.cc:302] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80e93d70a3734ec698623e810738cc00, f6458a1406e64303a885f09e9d64920f; no voters:
I20260807 08:33:28.588268 30673 raft_consensus.cc:2781] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:28.588357 30673 raft_consensus.cc:482] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:28.588389 30673 raft_consensus.cc:3037] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:28.589376 30673 raft_consensus.cc:504] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.589663 30673 leader_election.cc:288] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [CANDIDATE]: Term 1 election: Requested vote from peers 80e93d70a3734ec698623e810738cc00 (127.1.157.67:36431), 7243b68e40d24651b855d65120d7769d (127.1.157.65:42689)
I20260807 08:33:28.589942 30554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8e2df352f74e67995d3998bd3d6217" candidate_uuid: "f6458a1406e64303a885f09e9d64920f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e93d70a3734ec698623e810738cc00"
I20260807 08:33:28.589965 30143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8e2df352f74e67995d3998bd3d6217" candidate_uuid: "f6458a1406e64303a885f09e9d64920f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7243b68e40d24651b855d65120d7769d"
I20260807 08:33:28.590049 30143 raft_consensus.cc:3037] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:28.590054 30554 raft_consensus.cc:3037] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:28.590962 30143 raft_consensus.cc:2454] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6458a1406e64303a885f09e9d64920f in term 1.
I20260807 08:33:28.591035 30554 raft_consensus.cc:2454] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6458a1406e64303a885f09e9d64920f in term 1.
I20260807 08:33:28.591250 30243 leader_election.cc:302] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7243b68e40d24651b855d65120d7769d, f6458a1406e64303a885f09e9d64920f; no voters:
I20260807 08:33:28.591449 30673 raft_consensus.cc:2781] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:28.591643 30673 raft_consensus.cc:686] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 1 LEADER]: Becoming Leader. State: Replica: f6458a1406e64303a885f09e9d64920f, State: Running, Role: LEADER
I20260807 08:33:28.591910 30673 consensus_queue.cc:237] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } }
I20260807 08:33:28.594594 29909 catalog_manager.cc:5495] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f reported cstate change: term changed from 0 to 1, leader changed from <none> to f6458a1406e64303a885f09e9d64920f (127.1.157.66). New cstate: current_term: 1 leader_uuid: "f6458a1406e64303a885f09e9d64920f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6458a1406e64303a885f09e9d64920f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 44783 } health_report { overall_health: HEALTHY } } }
I20260807 08:33:28.603626 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.600672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.1.157.66:44783)
0807 08:33:28.600761 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:28.600772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:28.600784 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:33:28.600869 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:28.600871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:28.600916 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:28.600993 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:28.601177 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.601614 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.602397 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:28.602593 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:28.602659 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:28.602879 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:28.603210 (+ 331us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:28.603332 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:28.603498 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 08:33:28.603529 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 08:33:28.609788 30143 raft_consensus.cc:1261] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 1 FOLLOWER]: Refusing update from remote peer f6458a1406e64303a885f09e9d64920f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:28.609793 30554 raft_consensus.cc:1261] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 1 FOLLOWER]: Refusing update from remote peer f6458a1406e64303a885f09e9d64920f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:28.610369 30678 consensus_queue.cc:1035] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7243b68e40d24651b855d65120d7769d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:28.610527 30673 consensus_queue.cc:1035] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [LEADER]: Connected to new peer: Peer: permanent_uuid: "80e93d70a3734ec698623e810738cc00" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 36431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:28.616775 30681 mvcc.cc:205] Tried to move back new op lower bound from 7315831228858150912 to 7315831228794642432. Current Snapshot: MvccSnapshot[applied={T|T < 7315831228858150912}]
I20260807 08:33:28.617502 30682 mvcc.cc:205] Tried to move back new op lower bound from 7315831228858150912 to 7315831228794642432. Current Snapshot: MvccSnapshot[applied={T|T < 7315831228858150912}]
I20260807 08:33:28.624784 29918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:28.624903 29918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:28.626919 29918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2086a312c8e74d2dbcc0dbd1eadc656a: Sending DeleteTablet for 3 replicas of tablet bc8e2df352f74e67995d3998bd3d6217
I20260807 08:33:28.627508 30524 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8e2df352f74e67995d3998bd3d6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:28 UTC) from {username='slave'} at 127.0.0.1:50596
I20260807 08:33:28.627817 30329 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8e2df352f74e67995d3998bd3d6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:28 UTC) from {username='slave'} at 127.0.0.1:45766
I20260807 08:33:28.627806 30115 tablet_service.cc:1514] Processing DeleteTablet for tablet bc8e2df352f74e67995d3998bd3d6217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:28 UTC) from {username='slave'} at 127.0.0.1:57606
I20260807 08:33:28.628175 30695 tablet_replica.cc:330] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: stopping tablet replica
I20260807 08:33:28.628222 30697 tablet_replica.cc:330] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d: stopping tablet replica
I20260807 08:33:28.628440 30697 raft_consensus.cc:2229] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:28.628440 30695 raft_consensus.cc:2229] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:33:28.628705 30697 raft_consensus.cc:2258] T bc8e2df352f74e67995d3998bd3d6217 P 7243b68e40d24651b855d65120d7769d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:28.628755 30695 raft_consensus.cc:2258] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:28.628803 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 30004
I20260807 08:33:28.630170 30695 ts_tablet_manager.cc:1905] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:28.632488 30695 ts_tablet_manager.cc:1918] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:28.632580 30695 log.cc:1196] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/bc8e2df352f74e67995d3998bd3d6217
I20260807 08:33:28.632916 30695 ts_tablet_manager.cc:1939] T bc8e2df352f74e67995d3998bd3d6217 P f6458a1406e64303a885f09e9d64920f: Deleting consensus metadata
I20260807 08:33:28.634707 29902 catalog_manager.cc:4841] TS f6458a1406e64303a885f09e9d64920f (127.1.157.66:44783): tablet bc8e2df352f74e67995d3998bd3d6217 (table test-table [id=6b9c757e8e43454abb1b64f408f8305c]) successfully deleted
I20260807 08:33:28.635627 30694 tablet_replica.cc:330] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: stopping tablet replica
I20260807 08:33:28.635864 30694 raft_consensus.cc:2229] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:28.636091 30694 raft_consensus.cc:2258] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:28.637178 30694 ts_tablet_manager.cc:1905] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:28.639364 30694 ts_tablet_manager.cc:1918] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:28.639456 30694 log.cc:1196] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/bc8e2df352f74e67995d3998bd3d6217
I20260807 08:33:28.639698 30694 ts_tablet_manager.cc:1939] T bc8e2df352f74e67995d3998bd3d6217 P 80e93d70a3734ec698623e810738cc00: Deleting consensus metadata
I20260807 08:33:28.640591 29901 catalog_manager.cc:4841] TS 80e93d70a3734ec698623e810738cc00 (127.1.157.67:36431): tablet bc8e2df352f74e67995d3998bd3d6217 (table test-table [id=6b9c757e8e43454abb1b64f408f8305c]) successfully deleted
W20260807 08:33:28.642992 29902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:42689 (error 108)
I20260807 08:33:28.643756 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 30217
I20260807 08:33:28.643848 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:28.643605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:42689 (local address 127.0.0.1:57614)
0807 08:33:28.643721 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:28.643776 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:42689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:33:28.646790 29902 catalog_manager.cc:4584] TS 7243b68e40d24651b855d65120d7769d (127.1.157.65:42689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc8e2df352f74e67995d3998bd3d6217: Network error: Client connection negotiation failed: client connection to 127.1.157.65:42689: connect: Connection refused (error 111)
I20260807 08:33:28.656827 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 30422
I20260807 08:33:28.667753 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 29874
2026-08-07T08:33:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1051 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:28 dist-test-slave-6zbq krb5kdc[30705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:28 dist-test-slave-6zbq krb5kdc[30705](info): set up 2 sockets
Aug 07 08:33:28 dist-test-slave-6zbq krb5kdc[30705](info): commencing operation
krb5kdc: starting...
W20260807 08:33:29.781800 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:29 dist-test-slave-6zbq krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:29.891206 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44615
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:29.995157 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:29.995406 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:29.995448 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:29.999248 30725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:29.999289 30725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:29.999306 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:29.999321 30725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:29.999338 30725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:30.004218 30725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:44615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:30.005040 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:30.005934 30725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:30.013653 30733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.013774 30725 server_base.cc:1034] running on GCE node
W20260807 08:33:30.013645 30736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:30.013648 30734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.014492 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:30.015415 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:30.016563 30725 hybrid_clock.cc:648] HybridClock initialized: now 1786091610016561 us; error 43 us; skew 500 ppm
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:30.019043 30725 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:30.019799 30725 webserver.cc:460] Webserver started at http://127.1.157.126:33661/ using document root <none> and password file <none>
I20260807 08:33:30.020255 30725 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:30.020308 30725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:30.020464 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:30.022271 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "2889ea1d767d4e0ea364359f57c81ec7"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "e4edba7c34c7ab1da3ef40225c6ac0dd"
server_key_iv: "84e99c16b3cf241be9367f7b95bcc5b1"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.022799 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "2889ea1d767d4e0ea364359f57c81ec7"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "e4edba7c34c7ab1da3ef40225c6ac0dd"
server_key_iv: "84e99c16b3cf241be9367f7b95bcc5b1"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.026011 30725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:33:30.028119 30748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.029240 30725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:30.029373 30725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "2889ea1d767d4e0ea364359f57c81ec7"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "e4edba7c34c7ab1da3ef40225c6ac0dd"
server_key_iv: "84e99c16b3cf241be9367f7b95bcc5b1"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.029464 30725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:30.049587 30725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:30.052492 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:30.052670 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:30.059198 30725 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:44615
I20260807 08:33:30.059286 30828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:44615 every 8 connection(s)
I20260807 08:33:30.060086 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:30.062676 30830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:30.066403 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 30725
I20260807 08:33:30.066529 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:30.066803 1653 external_mini_cluster.cc:1428] Setting key cec790561eed813789c56a087640eaf7
I20260807 08:33:30.068332 30830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: Bootstrap starting.
I20260807 08:33:30.070665 30830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:30.071339 30830 log.cc:824] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:30.073271 30830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: No bootstrap required, opened a new log
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:30.077016 30830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } }
I20260807 08:33:30.077222 30830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:30.077288 30830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2889ea1d767d4e0ea364359f57c81ec7, State: Initialized, Role: FOLLOWER
I20260807 08:33:30.077665 30830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } }
I20260807 08:33:30.077797 30830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:30.077862 30830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:30.077957 30830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:30.078779 30830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } }
I20260807 08:33:30.079082 30830 leader_election.cc:302] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2889ea1d767d4e0ea364359f57c81ec7; no voters:
I20260807 08:33:30.079360 30830 leader_election.cc:288] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:30.079586 30837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:30.079823 30837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [term 1 LEADER]: Becoming Leader. State: Replica: 2889ea1d767d4e0ea364359f57c81ec7, State: Running, Role: LEADER
I20260807 08:33:30.080210 30837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } }
I20260807 08:33:30.080442 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.068085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.1.157.126:44615)
0807 08:33:30.068459 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.068479 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.068516 (+ 37us) server_negotiation.cc:407] Connection header received
0807 08:33:30.069134 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.069150 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.069458 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.069712 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:30.070601 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.071620 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.072783 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.073231 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.073373 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.075922 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:30.075927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:30.075939 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:30.075964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:30.077536 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.077954 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.077960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.077966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.078094 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.078375 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.078379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.078380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.078742 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:30.078927 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.079210 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.079529 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":97,"threads_started":1}
I20260807 08:33:30.080607 30830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:30.081868 30841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2889ea1d767d4e0ea364359f57c81ec7. Latest consensus state: current_term: 1 leader_uuid: "2889ea1d767d4e0ea364359f57c81ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } } }
I20260807 08:33:30.081890 30839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2889ea1d767d4e0ea364359f57c81ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2889ea1d767d4e0ea364359f57c81ec7" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 44615 } } }
I20260807 08:33:30.081988 30839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:30.081988 30841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:30.085012 30858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:30.085078 30858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:30.085165 30851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:30.085844 30851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:30.090642 30851 catalog_manager.cc:1347] Generated new cluster ID: 7d5ec6a0bf6047ceb4a351322da6594e
I20260807 08:33:30.090754 30851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:30.097344 30851 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:30.098104 30851 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:30.113595 30851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: Generated new TSK 0
I20260807 08:33:30.114257 30851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.157.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
Entry for principal kudu/127.1.157.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab.
I20260807 08:33:30.150069 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:30.256831 30868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:30.257041 30868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:30.257081 30868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:30.260926 30868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:30.260974 30868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:30.261026 30868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:30.265914 30868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.65.keytab
--principal=kudu/127.1.157.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:30.266736 30868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:30.267590 30868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:30.276171 30877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.276233 30868 server_base.cc:1034] running on GCE node
W20260807 08:33:30.276259 30879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:30.276259 30876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.276742 30868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:30.277299 30868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:30.278450 30868 hybrid_clock.cc:648] HybridClock initialized: now 1786091610278434 us; error 25 us; skew 500 ppm
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:30.280999 30868 init.cc:377] Logged in from keytab as kudu/127.1.157.65@KRBTEST.COM (short username kudu)
I20260807 08:33:30.281872 30868 webserver.cc:460] Webserver started at http://127.1.157.65:39633/ using document root <none> and password file <none>
I20260807 08:33:30.282403 30868 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:30.282471 30868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:30.282660 30868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:30.284504 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "ef8d88cb142f4fa48ca805252437c6fe"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "1ba8f40a9dc420674115f9a398a73e2a"
server_key_iv: "13a38e229e6be649b231a78cac52f642"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.285036 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "ef8d88cb142f4fa48ca805252437c6fe"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "1ba8f40a9dc420674115f9a398a73e2a"
server_key_iv: "13a38e229e6be649b231a78cac52f642"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.288363 30868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 08:33:30.290515 30891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.291569 30868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:30.291668 30868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "ef8d88cb142f4fa48ca805252437c6fe"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "1ba8f40a9dc420674115f9a398a73e2a"
server_key_iv: "13a38e229e6be649b231a78cac52f642"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.291774 30868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:30.303342 30868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:30.306272 30868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:30.306458 30868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:30.306988 30868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:30.307753 30868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:30.307802 30868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.307854 30868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:30.307883 30868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.316107 30868 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:35121
I20260807 08:33:30.316138 31059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:35121 every 8 connection(s)
I20260807 08:33:30.316901 30868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.65@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:30.325704 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 30868
I20260807 08:33:30.325837 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:30.326134 1653 external_mini_cluster.cc:1428] Setting key 3182de20b7ee0a4d6b3fd389b28d1400
I20260807 08:33:30.329212 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.318537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:40191 (local address 127.1.157.126:44615)
0807 08:33:30.318672 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.318676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.319468 (+ 792us) server_negotiation.cc:407] Connection header received
0807 08:33:30.320314 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.320317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.320375 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.320446 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:30.321624 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.322144 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.323257 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.323487 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.323576 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.325763 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:30.325767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:30.325769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:30.325794 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:30.327236 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.327795 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.327799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.327800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.327885 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.328198 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.328201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.328203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.328378 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:30.328464 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.328925 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.329046 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:33:30.329883 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.318900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44615 (local address 127.1.157.65:40191)
0807 08:33:30.319327 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.319367 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.320086 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.320617 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.320623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.320962 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:30.321472 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.321487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.322265 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.322269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.323129 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.323140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.323623 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.323628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.323787 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.324272 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:30.324287 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:30.325608 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:30.327374 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.327380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.327393 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.327697 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.327967 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.327970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.327972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.328104 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.328474 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:30.328481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:30.328706 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.328862 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.329150 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
I20260807 08:33:30.330962 31063 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44615
I20260807 08:33:30.331192 31063 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:30.331691 31063 heartbeater.cc:511] Master 127.1.157.126:44615 requested a full tablet report, sending...
I20260807 08:33:30.333205 30773 ts_manager.cc:194] Registered new tserver with Master: ef8d88cb142f4fa48ca805252437c6fe (127.1.157.65:35121)
I20260807 08:33:30.334280 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.65@KRBTEST.COM'} at 127.1.157.65:40191
WARNING: no policy specified for kudu/127.1.157.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
Entry for principal kudu/127.1.157.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab.
I20260807 08:33:30.350785 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:30.457760 31070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:30.458002 31070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:30.458042 31070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:30.461932 31070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:30.461978 31070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:30.462033 31070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:30.467018 31070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.66.keytab
--principal=kudu/127.1.157.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:30.467900 31070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:30.468797 31070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:30.477888 31080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.478010 31070 server_base.cc:1034] running on GCE node
W20260807 08:33:30.477931 31078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:30.477931 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.478535 31070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:30.479101 31070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:30.480269 31070 hybrid_clock.cc:648] HybridClock initialized: now 1786091610480252 us; error 26 us; skew 500 ppm
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:30.482955 31070 init.cc:377] Logged in from keytab as kudu/127.1.157.66@KRBTEST.COM (short username kudu)
I20260807 08:33:30.483716 31070 webserver.cc:460] Webserver started at http://127.1.157.66:46109/ using document root <none> and password file <none>
I20260807 08:33:30.484207 31070 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:30.484272 31070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:30.484460 31070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:30.486383 31070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "7f66ad2377864c53b7f6bc195778391e"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "769de6f4a29b2640d855da47d20316d4"
server_key_iv: "f5128dd35ebfc3b44dbb78b0f9878498"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.486920 31070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "7f66ad2377864c53b7f6bc195778391e"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "769de6f4a29b2640d855da47d20316d4"
server_key_iv: "f5128dd35ebfc3b44dbb78b0f9878498"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.490309 31070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:30.492565 31095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.493630 31070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:30.493742 31070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "7f66ad2377864c53b7f6bc195778391e"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "769de6f4a29b2640d855da47d20316d4"
server_key_iv: "f5128dd35ebfc3b44dbb78b0f9878498"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.493829 31070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:30.510044 31070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:30.512811 31070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:30.512972 31070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:30.513500 31070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:30.514271 31070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:30.514318 31070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.514427 31070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:30.514461 31070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.522469 31070 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:46591
I20260807 08:33:30.522521 31293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:46591 every 8 connection(s)
I20260807 08:33:30.523283 31070 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:30.526486 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 31070
I20260807 08:33:30.526603 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:30.526901 1653 external_mini_cluster.cc:1428] Setting key 5cb7ccde88b10c6af27ff06df8293cfe
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.66@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:30.536175 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.524887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:50951 (local address 127.1.157.126:44615)
0807 08:33:30.525002 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.525006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.525749 (+ 743us) server_negotiation.cc:407] Connection header received
0807 08:33:30.526647 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.526650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.526712 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.526815 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:30.528018 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.528678 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.529792 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.530094 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.530206 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.532760 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:30.532772 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:30.532775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:30.532802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:30.534177 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.534743 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.534746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.534747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.534838 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.535154 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.535156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.535157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.535331 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:30.535417 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.535895 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.536016 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.1.157.67@KRBTEST.COM; defaulting to no policy
I20260807 08:33:30.536934 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.525199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44615 (local address 127.1.157.66:50951)
0807 08:33:30.525604 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.525644 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.526430 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.526947 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.526955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.527344 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:30.527882 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.527899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.528806 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.528809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.529662 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.529671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.530267 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.530271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.530491 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.531055 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:30.531074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:30.532607 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:30.534289 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.534295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.534309 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.534647 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.534922 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.534924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.534926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.535058 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.535430 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:30.535436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:30.535664 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.535820 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.536119 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":97,"threads_started":1}
I20260807 08:33:30.537989 31294 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44615
I20260807 08:33:30.538208 31294 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:30.538697 31294 heartbeater.cc:511] Master 127.1.157.126:44615 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.67@KRBTEST.COM" created.
I20260807 08:33:30.539671 30773 ts_manager.cc:194] Registered new tserver with Master: 7f66ad2377864c53b7f6bc195778391e (127.1.157.66:46591)
I20260807 08:33:30.540202 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.66@KRBTEST.COM'} at 127.1.157.66:50951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
Entry for principal kudu/127.1.157.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab.
I20260807 08:33:30.552453 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:30.661152 31301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:30.661369 31301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:30.661407 31301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:30.665292 31301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:30.665340 31301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:30.665407 31301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:30.670403 31301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.67.keytab
--principal=kudu/127.1.157.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:44615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:30.671284 31301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:30.672190 31301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:30.681035 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:30.681062 31309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:30.681062 31308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:30.681263 31301 server_base.cc:1034] running on GCE node
I20260807 08:33:30.681749 31301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:30.682333 31301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:30.683525 31301 hybrid_clock.cc:648] HybridClock initialized: now 1786091610683494 us; error 43 us; skew 500 ppm
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:30.686038 31301 init.cc:377] Logged in from keytab as kudu/127.1.157.67@KRBTEST.COM (short username kudu)
I20260807 08:33:30.686838 31301 webserver.cc:460] Webserver started at http://127.1.157.67:41063/ using document root <none> and password file <none>
I20260807 08:33:30.687317 31301 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:30.687373 31301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:30.687530 31301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:30.689365 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "8aed3d032ad1415b9ba94dae810ca7fc"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "867237ab0bd6cb97dbd9cf448116ad5b"
server_key_iv: "0a696d6c06d169e3b821321845148f2a"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.689852 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "8aed3d032ad1415b9ba94dae810ca7fc"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "867237ab0bd6cb97dbd9cf448116ad5b"
server_key_iv: "0a696d6c06d169e3b821321845148f2a"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.693254 31301 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:33:30.695479 31322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.696636 31301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:33:30.696728 31301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "8aed3d032ad1415b9ba94dae810ca7fc"
format_stamp: "Formatted at 2026-08-07 08:33:30 on dist-test-slave-6zbq"
server_key: "867237ab0bd6cb97dbd9cf448116ad5b"
server_key_iv: "0a696d6c06d169e3b821321845148f2a"
server_key_version: "encryptionkey@0"
I20260807 08:33:30.696808 31301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:30.713310 31301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:30.715790 31301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:30.715926 31301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:30.716416 31301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:30.717126 31301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:30.717172 31301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.717208 31301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:30.717228 31301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:30.725139 31301 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:34641
I20260807 08:33:30.725174 31500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:34641 every 8 connection(s)
I20260807 08:33:30.725926 31301 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:33:30.727797 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 31301
I20260807 08:33:30.727903 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:30.728178 1653 external_mini_cluster.cc:1428] Setting key ac581d8121fce1bdf1f3e56eab3c8771
Aug 07 08:33:30 dist-test-slave-6zbq krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.67@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:30.737443 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.727510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:58131 (local address 127.1.157.126:44615)
0807 08:33:30.727671 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.727676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.728477 (+ 801us) server_negotiation.cc:407] Connection header received
0807 08:33:30.729483 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.729486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.729535 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.729599 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:30.730871 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.731383 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.732398 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.732605 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.732697 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.734863 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:30.734866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:30.734868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:30.734891 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:30.735927 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.736352 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.736354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.736355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.736420 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.736658 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.736660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.736661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.736813 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:30.736882 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.737247 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.737333 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 08:33:30.738123 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.727822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:44615 (local address 127.1.157.67:58131)
0807 08:33:30.728320 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.728373 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.729287 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.729794 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.729803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.730255 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:30.730755 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.730768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.731468 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.731471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.732311 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.732319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.732729 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.732731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.732858 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.733374 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:30.733389 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:30.734719 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:30.736027 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.736032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.736041 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.736281 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.736480 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:30.736482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:30.736484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:30.736598 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:30.736881 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:30.736885 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:30.737078 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.737215 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.737441 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":131,"threads_started":1}
I20260807 08:33:30.739029 31501 heartbeater.cc:347] Connected to a master server at 127.1.157.126:44615
I20260807 08:33:30.739228 31501 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:30.739642 31501 heartbeater.cc:511] Master 127.1.157.126:44615 requested a full tablet report, sending...
I20260807 08:33:30.740561 30773 ts_manager.cc:194] Registered new tserver with Master: 8aed3d032ad1415b9ba94dae810ca7fc (127.1.157.67:34641)
I20260807 08:33:30.741153 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.157.67@KRBTEST.COM'} at 127.1.157.67:58131
I20260807 08:33:30.741648 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:30.749696 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.743461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.1.157.126:44615)
0807 08:33:30.743567 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.743571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.743735 (+ 164us) server_negotiation.cc:407] Connection header received
0807 08:33:30.743836 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.743839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.743877 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.743936 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:30.744560 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.745035 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.745827 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.746030 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.746110 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.747259 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:30.747263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:30.747265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:30.747288 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:30.748246 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.748602 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.748606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.748607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.748683 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:30.748918 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:30.748921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:30.748923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:30.749117 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:30.749242 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.749454 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.749560 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:33:30.752125 30771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:30.754945 30771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:30.768097 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.763116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:34641 (local address 127.0.0.1:38074)
0807 08:33:30.763326 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.763337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.763400 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.764198 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.764201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.764216 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:30.764314 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.764321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765840 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.765843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.766756 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.766765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767607 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.767715 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.767753 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.767852 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.767959 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":6,"thread_start_us":91,"threads_started":1}
I20260807 08:33:30.768150 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.763493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.1.157.67:34641)
0807 08:33:30.763805 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.763815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.763833 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:33:30.763894 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.763899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.764054 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.764437 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:30.764562 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765672 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.766888 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767493 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767590 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.767736 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.767916 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.768015 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":128,"threads_started":1}
I20260807 08:33:30.768126 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.762553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:35121 (local address 127.0.0.1:43816)
0807 08:33:30.763239 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.763251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.763331 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.764227 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.764230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.764247 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:30.764385 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.764393 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765344 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.765347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.766681 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.766691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767814 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.767896 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.767916 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.767976 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.768032 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":176,"thread_start_us":89,"threads_started":1}
I20260807 08:33:30.768442 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.763200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.1.157.65:35121)
0807 08:33:30.763618 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.763629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.763652 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:33:30.763725 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.763730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.763915 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.764056 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:30.764504 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765222 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.766808 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767577 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767694 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.767823 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.768217 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.768312 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":128,"threads_started":1}
I20260807 08:33:30.768350 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.762428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:46591 (local address 127.0.0.1:42994)
0807 08:33:30.762732 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.762784 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.762899 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.764054 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.764061 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.764083 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:30.764228 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.764236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765813 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.765816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.766954 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.766963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767988 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.768078 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.768101 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.768165 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.768232 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":25,"thread_start_us":70,"threads_started":1}
I20260807 08:33:30.768839 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.762844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.1.157.66:46591)
0807 08:33:30.763562 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.763572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.763590 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:33:30.763656 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.763661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.763887 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.764372 (+ 485us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:30.764528 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.765672 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.767076 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.767861 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.768431 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.768567 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.768617 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.768703 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":122,"threads_started":1}
I20260807 08:33:30.771363 30963 tablet_service.cc:1467] Processing CreateTablet for tablet 927bc3aab17040a991f8d7f02aca8298 (DEFAULT_TABLE table=test-table [id=a27d0fd2a03a42cfb43eede66ad1a29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:30.771363 31402 tablet_service.cc:1467] Processing CreateTablet for tablet 927bc3aab17040a991f8d7f02aca8298 (DEFAULT_TABLE table=test-table [id=a27d0fd2a03a42cfb43eede66ad1a29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:30.771700 31193 tablet_service.cc:1467] Processing CreateTablet for tablet 927bc3aab17040a991f8d7f02aca8298 (DEFAULT_TABLE table=test-table [id=a27d0fd2a03a42cfb43eede66ad1a29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:30.772429 30963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927bc3aab17040a991f8d7f02aca8298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:30.772437 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927bc3aab17040a991f8d7f02aca8298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:30.772534 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927bc3aab17040a991f8d7f02aca8298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:30.778434 31521 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Bootstrap starting.
I20260807 08:33:30.779613 31523 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Bootstrap starting.
I20260807 08:33:30.779979 31525 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Bootstrap starting.
I20260807 08:33:30.781365 31521 tablet_bootstrap.cc:654] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:30.781667 31523 tablet_bootstrap.cc:654] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:30.782056 31525 tablet_bootstrap.cc:654] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:30.782202 31521 log.cc:824] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:30.782989 31523 log.cc:824] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:30.783169 31525 log.cc:824] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:30.784116 31521 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: No bootstrap required, opened a new log
I20260807 08:33:30.784305 31521 ts_tablet_manager.cc:1397] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 08:33:30.784775 31525 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: No bootstrap required, opened a new log
I20260807 08:33:30.784857 31523 tablet_bootstrap.cc:492] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: No bootstrap required, opened a new log
I20260807 08:33:30.785012 31525 ts_tablet_manager.cc:1397] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 08:33:30.785029 31523 ts_tablet_manager.cc:1397] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:33:30.787695 31525 raft_consensus.cc:348] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.787901 31525 raft_consensus.cc:374] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:30.787969 31525 raft_consensus.cc:729] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aed3d032ad1415b9ba94dae810ca7fc, State: Initialized, Role: FOLLOWER
I20260807 08:33:30.787995 31521 raft_consensus.cc:348] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.788290 31521 raft_consensus.cc:374] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:30.788355 31521 raft_consensus.cc:729] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f66ad2377864c53b7f6bc195778391e, State: Initialized, Role: FOLLOWER
I20260807 08:33:30.788393 31525 consensus_queue.cc:260] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.788571 31523 raft_consensus.cc:348] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.788797 31521 consensus_queue.cc:260] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.788866 31523 raft_consensus.cc:374] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:30.788916 31523 raft_consensus.cc:729] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8d88cb142f4fa48ca805252437c6fe, State: Initialized, Role: FOLLOWER
I20260807 08:33:30.788965 31501 heartbeater.cc:503] Master 127.1.157.126:44615 was elected leader, sending a full tablet report...
I20260807 08:33:30.789312 31525 ts_tablet_manager.cc:1428] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 08:33:30.789388 31523 consensus_queue.cc:260] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.789764 31521 ts_tablet_manager.cc:1428] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 08:33:30.789983 31063 heartbeater.cc:503] Master 127.1.157.126:44615 was elected leader, sending a full tablet report...
I20260807 08:33:30.790055 31294 heartbeater.cc:503] Master 127.1.157.126:44615 was elected leader, sending a full tablet report...
I20260807 08:33:30.790241 31523 ts_tablet_manager.cc:1428] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 08:33:30.817929 31064 tablet.cc:2369] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:30.932413 31532 raft_consensus.cc:482] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:30.932633 31532 raft_consensus.cc:504] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.933868 31532 leader_election.cc:288] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8d88cb142f4fa48ca805252437c6fe (127.1.157.65:35121), 8aed3d032ad1415b9ba94dae810ca7fc (127.1.157.67:34641)
I20260807 08:33:30.937407 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.934097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:35121 (local address 127.1.157.66:44463)
0807 08:33:30.934231 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.934247 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.934366 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.934738 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.934741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.934759 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:30.934868 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.934873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.935589 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.935592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.936369 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.936378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.937011 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.937014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.937120 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.937138 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.937203 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.937277 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 08:33:30.937417 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.934160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:44463 (local address 127.1.157.65:35121)
0807 08:33:30.934315 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.934320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.934334 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:33:30.934475 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.934479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.934527 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.934591 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:30.934979 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.935482 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.936495 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.936891 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.937007 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.937061 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.937233 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.937314 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 08:33:30.937788 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.934384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:34641 (local address 127.1.157.66:58957)
0807 08:33:30.934747 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:30.934758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:30.934865 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:30.935094 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:30.935096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:30.935111 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:30.935229 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.935235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.935948 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.935951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.936719 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:30.936728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.937367 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.937371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:30.937466 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.937489 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:30.937592 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:33:30.937651 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":131,"threads_started":1}
I20260807 08:33:30.937831 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.934537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:58957 (local address 127.1.157.67:34641)
0807 08:33:30.934678 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.934683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.934845 (+ 162us) server_negotiation.cc:407] Connection header received
0807 08:33:30.934968 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.934971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.935012 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.935073 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:30.935335 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.935841 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.936842 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.937256 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.937360 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.937408 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.937635 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.937717 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:33:30.938090 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927bc3aab17040a991f8d7f02aca8298" candidate_uuid: "7f66ad2377864c53b7f6bc195778391e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8d88cb142f4fa48ca805252437c6fe" is_pre_election: true
I20260807 08:33:30.938445 30997 raft_consensus.cc:2454] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f66ad2377864c53b7f6bc195778391e in term 0.
I20260807 08:33:30.938549 31429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927bc3aab17040a991f8d7f02aca8298" candidate_uuid: "7f66ad2377864c53b7f6bc195778391e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" is_pre_election: true
I20260807 08:33:30.938818 31429 raft_consensus.cc:2454] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f66ad2377864c53b7f6bc195778391e in term 0.
I20260807 08:33:30.938865 31105 leader_election.cc:302] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f66ad2377864c53b7f6bc195778391e, ef8d88cb142f4fa48ca805252437c6fe; no voters:
I20260807 08:33:30.939123 31532 raft_consensus.cc:2781] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:30.939200 31532 raft_consensus.cc:482] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:30.939229 31532 raft_consensus.cc:3037] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:30.940243 31532 raft_consensus.cc:504] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.940554 31532 leader_election.cc:288] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [CANDIDATE]: Term 1 election: Requested vote from peers ef8d88cb142f4fa48ca805252437c6fe (127.1.157.65:35121), 8aed3d032ad1415b9ba94dae810ca7fc (127.1.157.67:34641)
I20260807 08:33:30.940941 31429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927bc3aab17040a991f8d7f02aca8298" candidate_uuid: "7f66ad2377864c53b7f6bc195778391e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aed3d032ad1415b9ba94dae810ca7fc"
I20260807 08:33:30.940943 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927bc3aab17040a991f8d7f02aca8298" candidate_uuid: "7f66ad2377864c53b7f6bc195778391e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8d88cb142f4fa48ca805252437c6fe"
I20260807 08:33:30.941063 31429 raft_consensus.cc:3037] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:30.941063 30997 raft_consensus.cc:3037] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:30.942315 31429 raft_consensus.cc:2454] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f66ad2377864c53b7f6bc195778391e in term 1.
I20260807 08:33:30.942319 30997 raft_consensus.cc:2454] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f66ad2377864c53b7f6bc195778391e in term 1.
I20260807 08:33:30.942629 31105 leader_election.cc:302] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f66ad2377864c53b7f6bc195778391e, ef8d88cb142f4fa48ca805252437c6fe; no voters:
I20260807 08:33:30.942800 31532 raft_consensus.cc:2781] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:30.943006 31532 raft_consensus.cc:686] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 1 LEADER]: Becoming Leader. State: Replica: 7f66ad2377864c53b7f6bc195778391e, State: Running, Role: LEADER
I20260807 08:33:30.943323 31532 consensus_queue.cc:237] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } }
I20260807 08:33:30.946261 30770 catalog_manager.cc:5495] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f66ad2377864c53b7f6bc195778391e (127.1.157.66). New cstate: current_term: 1 leader_uuid: "7f66ad2377864c53b7f6bc195778391e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f66ad2377864c53b7f6bc195778391e" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 46591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 } health_report { overall_health: UNKNOWN } } }
W20260807 08:33:30.976811 31502 tablet.cc:2369] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:31.000092 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:30.996539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.1.157.66:46591)
0807 08:33:30.996666 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:30.996670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:30.996684 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:33:30.996820 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:30.996822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:30.996910 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:30.996997 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:30.997228 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.997774 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.998709 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:30.998924 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:30.999009 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:30.999291 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:30.999723 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:30.999824 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:30.999893 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 08:33:30.999947 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:33:31.005513 31429 raft_consensus.cc:1261] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 1 FOLLOWER]: Refusing update from remote peer 7f66ad2377864c53b7f6bc195778391e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:31.005546 30997 raft_consensus.cc:1261] T 927bc3aab17040a991f8d7f02aca8298 P ef8d88cb142f4fa48ca805252437c6fe [term 1 FOLLOWER]: Refusing update from remote peer 7f66ad2377864c53b7f6bc195778391e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:31.006138 31532 consensus_queue.cc:1035] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8d88cb142f4fa48ca805252437c6fe" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 35121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:31.006366 31537 consensus_queue.cc:1035] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aed3d032ad1415b9ba94dae810ca7fc" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:31.013528 31544 mvcc.cc:205] Tried to move back new op lower bound from 7315831238671941632 to 7315831238426173440. Current Snapshot: MvccSnapshot[applied={T|T < 7315831238671941632}]
I20260807 08:33:31.015671 31543 mvcc.cc:205] Tried to move back new op lower bound from 7315831238671941632 to 7315831238426173440. Current Snapshot: MvccSnapshot[applied={T|T < 7315831238671941632}]
I20260807 08:33:31.021088 30771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:31.021248 30771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:31.023416 30771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2889ea1d767d4e0ea364359f57c81ec7: Sending DeleteTablet for 3 replicas of tablet 927bc3aab17040a991f8d7f02aca8298
I20260807 08:33:31.024067 31193 tablet_service.cc:1514] Processing DeleteTablet for tablet 927bc3aab17040a991f8d7f02aca8298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:31 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:42994
I20260807 08:33:31.024217 31402 tablet_service.cc:1514] Processing DeleteTablet for tablet 927bc3aab17040a991f8d7f02aca8298 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:31 UTC) from {username='kudu', principal='kudu/127.1.157.126@KRBTEST.COM'} at 127.0.0.1:38074
I20260807 08:33:31.024647 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 30868
I20260807 08:33:31.025276 31563 tablet_replica.cc:330] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: stopping tablet replica
I20260807 08:33:31.025288 31561 tablet_replica.cc:330] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: stopping tablet replica
I20260807 08:33:31.025610 31561 raft_consensus.cc:2229] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:33:31.025629 31563 raft_consensus.cc:2229] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:31.025934 31563 raft_consensus.cc:2258] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:31.026055 31561 raft_consensus.cc:2258] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:31.030663 31561 ts_tablet_manager.cc:1905] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:31.030788 31563 ts_tablet_manager.cc:1905] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:31.034139 31561 ts_tablet_manager.cc:1918] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:31.034363 31561 log.cc:1196] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/927bc3aab17040a991f8d7f02aca8298
I20260807 08:33:31.034482 31563 ts_tablet_manager.cc:1918] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:31.034647 31563 log.cc:1196] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/927bc3aab17040a991f8d7f02aca8298
I20260807 08:33:31.034826 31561 ts_tablet_manager.cc:1939] T 927bc3aab17040a991f8d7f02aca8298 P 7f66ad2377864c53b7f6bc195778391e: Deleting consensus metadata
I20260807 08:33:31.035025 31563 ts_tablet_manager.cc:1939] T 927bc3aab17040a991f8d7f02aca8298 P 8aed3d032ad1415b9ba94dae810ca7fc: Deleting consensus metadata
I20260807 08:33:31.036193 30753 catalog_manager.cc:4841] TS 8aed3d032ad1415b9ba94dae810ca7fc (127.1.157.67:34641): tablet 927bc3aab17040a991f8d7f02aca8298 (table test-table [id=a27d0fd2a03a42cfb43eede66ad1a29d]) successfully deleted
I20260807 08:33:31.036223 30750 catalog_manager.cc:4841] TS 7f66ad2377864c53b7f6bc195778391e (127.1.157.66:46591): tablet 927bc3aab17040a991f8d7f02aca8298 (table test-table [id=a27d0fd2a03a42cfb43eede66ad1a29d]) successfully deleted
W20260807 08:33:31.043087 30753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:35121 (error 108)
I20260807 08:33:31.043921 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 31070
I20260807 08:33:31.044133 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.043835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:35121 (local address 127.0.0.1:43832)
0807 08:33:31.043972 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.044047 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 08:33:31.050143 30753 catalog_manager.cc:4584] TS ef8d88cb142f4fa48ca805252437c6fe (127.1.157.65:35121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 927bc3aab17040a991f8d7f02aca8298: Network error: Client connection negotiation failed: client connection to 127.1.157.65:35121: connect: Connection refused (error 111)
I20260807 08:33:31.055169 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 31301
I20260807 08:33:31.063685 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 30725
2026-08-07T08:33:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2397 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T08:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:31Z Disabled control of system clock
I20260807 08:33:31.096952 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46695
--webserver_interface=127.1.157.126
--webserver_port=0
--builtin_ntp_servers=127.1.157.84:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:46695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:31.200973 31570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:31.201206 31570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:31.201239 31570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:31.205034 31570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:31.205076 31570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:31.205090 31570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:31.205104 31570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:31.205117 31570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:31.209551 31570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:46695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:31.210390 31570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:31.211583 31570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:33:31.219290 31570 server_base.cc:1034] running on GCE node
W20260807 08:33:31.219314 31576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.219318 31577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.219442 31580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:31.220014 31570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:31.220968 31570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:31.222117 31570 hybrid_clock.cc:648] HybridClock initialized: now 1786091611222098 us; error 33 us; skew 500 ppm
I20260807 08:33:31.223860 31570 webserver.cc:460] Webserver started at http://127.1.157.126:33267/ using document root <none> and password file <none>
I20260807 08:33:31.224359 31570 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:31.224421 31570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:31.224607 31570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:31.226495 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "88fc73bbc5a54a39a58123eeb5f3d670"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "4132aead3abe7c3db51bdbf2a0c38b94"
server_key_iv: "f6b70f7e494fa2e8b1fa514e77b47e18"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.227013 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "88fc73bbc5a54a39a58123eeb5f3d670"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "4132aead3abe7c3db51bdbf2a0c38b94"
server_key_iv: "f6b70f7e494fa2e8b1fa514e77b47e18"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.230242 31570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:33:31.232414 31591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.233515 31570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:31.233623 31570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "88fc73bbc5a54a39a58123eeb5f3d670"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "4132aead3abe7c3db51bdbf2a0c38b94"
server_key_iv: "f6b70f7e494fa2e8b1fa514e77b47e18"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.233708 31570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:31.250469 31570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:31.251020 31570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:31.251178 31570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:31.258159 31674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:46695 every 8 connection(s)
I20260807 08:33:31.258157 31570 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:46695
I20260807 08:33:31.259042 31570 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:31.261791 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 31570
I20260807 08:33:31.261822 31675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:31.261900 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:31.262130 1653 external_mini_cluster.cc:1428] Setting key 6b188487109456179f31f1d88ae9a1be
I20260807 08:33:31.267725 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: Bootstrap starting.
I20260807 08:33:31.269683 31675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:31.270442 31675 log.cc:824] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:31.270728 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.263366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.1.157.126:46695)
0807 08:33:31.263973 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.263989 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.264020 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:33:31.264717 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.264732 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.265078 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.265382 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:31.265776 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.266862 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.267724 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.268217 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.268360 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.269147 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:31.269152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:31.269164 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:31.269497 (+ 333us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:31.269550 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:31.269568 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:31.269652 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.269789 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.270019 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
I20260807 08:33:31.272570 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: No bootstrap required, opened a new log
I20260807 08:33:31.276057 31675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } }
I20260807 08:33:31.276265 31675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:31.276317 31675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88fc73bbc5a54a39a58123eeb5f3d670, State: Initialized, Role: FOLLOWER
I20260807 08:33:31.276749 31675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } }
I20260807 08:33:31.276881 31675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:31.276925 31675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:31.276993 31675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:31.277741 31675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } }
I20260807 08:33:31.278009 31675 leader_election.cc:302] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88fc73bbc5a54a39a58123eeb5f3d670; no voters:
I20260807 08:33:31.278240 31675 leader_election.cc:288] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:31.278384 31682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:31.278579 31682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [term 1 LEADER]: Becoming Leader. State: Replica: 88fc73bbc5a54a39a58123eeb5f3d670, State: Running, Role: LEADER
I20260807 08:33:31.278877 31682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } }
I20260807 08:33:31.279270 31675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:31.280417 31683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } } }
I20260807 08:33:31.280575 31683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:31.280417 31684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88fc73bbc5a54a39a58123eeb5f3d670. Latest consensus state: current_term: 1 leader_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88fc73bbc5a54a39a58123eeb5f3d670" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46695 } } }
I20260807 08:33:31.280666 31684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:31.283937 31700 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:31.284004 31700 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:31.284086 31695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:31.284727 31695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:31.289134 31695 catalog_manager.cc:1347] Generated new cluster ID: aa02f116ba994c4590c93c3b72ecbe85
I20260807 08:33:31.289191 31695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:31.323843 31695 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:31.325027 31695 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:31.341353 31695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: Generated new TSK 0
I20260807 08:33:31.342139 31695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:31.347867 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.65:0
--local_ip_for_outbound_sockets=127.1.157.65
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--builtin_ntp_servers=127.1.157.84:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 08:33:31.453895 31705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:31.454119 31705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:31.454167 31705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:31.458021 31705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:31.458069 31705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:31.458137 31705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.65
I20260807 08:33:31.462692 31705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.157.65
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.65
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:31.463560 31705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:31.464775 31705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:31.473637 31723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.473645 31719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:31.473645 31705 server_base.cc:1034] running on GCE node
W20260807 08:33:31.473661 31717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:31.474275 31705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:31.474888 31705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:31.476044 31705 hybrid_clock.cc:648] HybridClock initialized: now 1786091611476030 us; error 28 us; skew 500 ppm
I20260807 08:33:31.477730 31705 webserver.cc:460] Webserver started at http://127.1.157.65:36709/ using document root <none> and password file <none>
I20260807 08:33:31.478243 31705 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:31.478317 31705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:31.478518 31705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:31.480481 31705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data/instance:
uuid: "2ee22176403b479a8e2e08c6d9d2158d"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "205aad7985dc554fc90cec978e3a0e7e"
server_key_iv: "fb1b588dd34c2b730fff17100b1d9c80"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.480988 31705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance:
uuid: "2ee22176403b479a8e2e08c6d9d2158d"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "205aad7985dc554fc90cec978e3a0e7e"
server_key_iv: "fb1b588dd34c2b730fff17100b1d9c80"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.484335 31705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 08:33:31.486496 31731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.487584 31705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:31.487685 31705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal
uuid: "2ee22176403b479a8e2e08c6d9d2158d"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "205aad7985dc554fc90cec978e3a0e7e"
server_key_iv: "fb1b588dd34c2b730fff17100b1d9c80"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.487769 31705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:31.508559 31705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:31.509101 31705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:31.509253 31705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:31.509756 31705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:31.510603 31705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:31.510650 31705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.510699 31705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:31.510726 31705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.518914 31705 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.65:41959
I20260807 08:33:31.518940 31914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.65:41959 every 8 connection(s)
I20260807 08:33:31.519681 31705 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/data/info.pb
I20260807 08:33:31.524221 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 31705
I20260807 08:33:31.524338 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-0/wal/instance
I20260807 08:33:31.524606 1653 external_mini_cluster.cc:1428] Setting key 0a708753aff67f65e326c6bda4102454
I20260807 08:33:31.526646 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.66:0
--local_ip_for_outbound_sockets=127.1.157.66
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--builtin_ntp_servers=127.1.157.84:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:31.528337 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.521242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.65:58523 (local address 127.1.157.126:46695)
0807 08:33:31.521370 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.521374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.522304 (+ 930us) server_negotiation.cc:407] Connection header received
0807 08:33:31.523563 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.523566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.523610 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.523681 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:31.524477 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.525166 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.526522 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.526856 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.526961 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.527881 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:31.527885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:31.527888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:31.527901 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:31.527922 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:31.527931 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:31.527997 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.528132 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.528199 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:33:31.529017 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.521592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46695 (local address 127.1.157.65:58523)
0807 08:33:31.522157 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.522205 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.523393 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.523817 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.523822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.523897 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:31.524336 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.524353 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.525294 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.525298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.526409 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.526419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.526994 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.526997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.527127 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.527646 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:31.527648 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:31.527683 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:31.527742 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:31.528007 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:31.528012 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:31.528015 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.528106 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.528364 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":108,"threads_started":1}
I20260807 08:33:31.530002 31915 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46695
I20260807 08:33:31.530215 31915 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:31.530745 31915 heartbeater.cc:511] Master 127.1.157.126:46695 requested a full tablet report, sending...
I20260807 08:33:31.532119 31617 ts_manager.cc:194] Registered new tserver with Master: 2ee22176403b479a8e2e08c6d9d2158d (127.1.157.65:41959)
I20260807 08:33:31.533170 31617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.65:58523
W20260807 08:33:31.633208 31919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:31.633423 31919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:31.633460 31919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:31.637365 31919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:31.637408 31919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:31.637470 31919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.66
I20260807 08:33:31.642017 31919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.157.66
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.66
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:31.642925 31919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:31.644146 31919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:31.652653 31928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.652670 31931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.652670 31929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:31.652834 31919 server_base.cc:1034] running on GCE node
I20260807 08:33:31.653280 31919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:31.653836 31919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:31.654978 31919 hybrid_clock.cc:648] HybridClock initialized: now 1786091611654955 us; error 41 us; skew 500 ppm
I20260807 08:33:31.656617 31919 webserver.cc:460] Webserver started at http://127.1.157.66:35795/ using document root <none> and password file <none>
I20260807 08:33:31.657114 31919 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:31.657176 31919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:31.657372 31919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:31.659261 31919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data/instance:
uuid: "aa77a2ebdb964f2a9394bd558451f97f"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "ca56acaa560df239ae14cf0c40db367e"
server_key_iv: "6119e58d2c72580d736cfd28c2fa8ab7"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.659761 31919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance:
uuid: "aa77a2ebdb964f2a9394bd558451f97f"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "ca56acaa560df239ae14cf0c40db367e"
server_key_iv: "6119e58d2c72580d736cfd28c2fa8ab7"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.663069 31919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:31.665144 31942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.666172 31919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:31.666285 31919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal
uuid: "aa77a2ebdb964f2a9394bd558451f97f"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "ca56acaa560df239ae14cf0c40db367e"
server_key_iv: "6119e58d2c72580d736cfd28c2fa8ab7"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.666404 31919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:31.676939 31919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:31.677459 31919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:31.677603 31919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:31.678089 31919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:31.678962 31919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:31.679008 31919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.679056 31919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:31.679086 31919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.687089 31919 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.66:36021
I20260807 08:33:31.687258 32120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.66:36021 every 8 connection(s)
I20260807 08:33:31.687893 31919 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/data/info.pb
I20260807 08:33:31.693552 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 31919
I20260807 08:33:31.693693 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal/instance
I20260807 08:33:31.693976 1653 external_mini_cluster.cc:1428] Setting key e07c86807c27d813843ee5266af11c54
I20260807 08:33:31.695830 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.157.67:0
--local_ip_for_outbound_sockets=127.1.157.67
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--builtin_ntp_servers=127.1.157.84:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 08:33:31.696579 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.689496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:59911 (local address 127.1.157.126:46695)
0807 08:33:31.689616 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.689620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.690615 (+ 995us) server_negotiation.cc:407] Connection header received
0807 08:33:31.691953 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.691956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.692003 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.692072 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:31.692835 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.693339 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.694774 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.694996 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.695080 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.696150 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:31.696154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:31.696156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:31.696169 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:31.696192 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:31.696202 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:31.696274 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.696379 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.696440 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:33:31.697302 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.689840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46695 (local address 127.1.157.66:59911)
0807 08:33:31.690486 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.690525 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.691777 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.692203 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.692209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.692284 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:31.692726 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.692738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.693462 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.693465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.694646 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.694657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.695136 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.695139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.695335 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.695870 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:31.695872 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:31.695908 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:31.695977 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:31.696270 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:31.696275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:31.696278 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.696360 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.696563 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":106,"threads_started":1}
I20260807 08:33:31.698284 32121 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46695
I20260807 08:33:31.698532 32121 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:31.699057 32121 heartbeater.cc:511] Master 127.1.157.126:46695 requested a full tablet report, sending...
I20260807 08:33:31.700042 31617 ts_manager.cc:194] Registered new tserver with Master: aa77a2ebdb964f2a9394bd558451f97f (127.1.157.66:36021)
I20260807 08:33:31.700794 31617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.66:59911
W20260807 08:33:31.802804 32124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:31.803020 32124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:31.803053 32124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:31.806834 32124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:31.806874 32124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:31.806923 32124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.157.67
I20260807 08:33:31.811298 32124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.157.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.157.67
--webserver_port=0
--tserver_master_addrs=127.1.157.126:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.157.67
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:31.812105 32124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:31.813256 32124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:31.822515 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.822510 32134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:31.822518 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:31.822741 32124 server_base.cc:1034] running on GCE node
I20260807 08:33:31.823115 32124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:31.823649 32124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:31.824806 32124 hybrid_clock.cc:648] HybridClock initialized: now 1786091611824793 us; error 27 us; skew 500 ppm
I20260807 08:33:31.826454 32124 webserver.cc:460] Webserver started at http://127.1.157.67:39861/ using document root <none> and password file <none>
I20260807 08:33:31.826957 32124 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:31.827021 32124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:31.827198 32124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:31.829046 32124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data/instance:
uuid: "c06f1d619cbe4a8cb0901462c07a1871"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "69da09bed060b418acdff563b0766829"
server_key_iv: "4cfda89690181e0ac763a7fcb0ff921a"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.829557 32124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance:
uuid: "c06f1d619cbe4a8cb0901462c07a1871"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "69da09bed060b418acdff563b0766829"
server_key_iv: "4cfda89690181e0ac763a7fcb0ff921a"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.832968 32124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260807 08:33:31.835199 32144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.836254 32124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 08:33:31.836364 32124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal
uuid: "c06f1d619cbe4a8cb0901462c07a1871"
format_stamp: "Formatted at 2026-08-07 08:33:31 on dist-test-slave-6zbq"
server_key: "69da09bed060b418acdff563b0766829"
server_key_iv: "4cfda89690181e0ac763a7fcb0ff921a"
server_key_version: "encryptionkey@0"
I20260807 08:33:31.836464 32124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:31.853194 32124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:31.853719 32124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:31.853847 32124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:31.854312 32124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 08:33:31.855171 32124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 08:33:31.855217 32124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.855252 32124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 08:33:31.855281 32124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:31.863304 32124 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.67:40547
I20260807 08:33:31.863351 32309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.67:40547 every 8 connection(s)
I20260807 08:33:31.864101 32124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/data/info.pb
I20260807 08:33:31.872465 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.865672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.67:60257 (local address 127.1.157.126:46695)
0807 08:33:31.865797 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.865801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.866760 (+ 959us) server_negotiation.cc:407] Connection header received
0807 08:33:31.868057 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.868060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.868119 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.868199 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:31.869020 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.869547 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.870734 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.870956 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.871045 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.872012 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:31.872016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:31.872018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:31.872030 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:31.872049 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:31.872057 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:31.872115 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.872260 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.872322 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 08:33:31.872643 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 32124
I20260807 08:33:31.872750 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal/instance
I20260807 08:33:31.872991 1653 external_mini_cluster.cc:1428] Setting key 43f02394fa4a9e3286f5df499a5c4203
I20260807 08:33:31.873370 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.866001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:46695 (local address 127.1.157.67:60257)
0807 08:33:31.866622 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.866659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.867864 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.868357 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.868363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.868442 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:31.868887 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.868899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.869701 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.869705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.870611 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.870621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.871093 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.871097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.871244 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.871768 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 08:33:31.871769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:31.871807 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 08:33:31.871866 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:31.872126 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:31.872135 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:31.872141 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.872244 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.872490 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260807 08:33:31.874552 32310 heartbeater.cc:347] Connected to a master server at 127.1.157.126:46695
I20260807 08:33:31.874764 32310 heartbeater.cc:464] Registering TS with master...
I20260807 08:33:31.875312 32310 heartbeater.cc:511] Master 127.1.157.126:46695 requested a full tablet report, sending...
I20260807 08:33:31.876209 31617 ts_manager.cc:194] Registered new tserver with Master: c06f1d619cbe4a8cb0901462c07a1871 (127.1.157.67:40547)
I20260807 08:33:31.876746 31617 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.157.67:60257
I20260807 08:33:31.886442 1653 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 08:33:31.891759 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.888223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.1.157.126:46695)
0807 08:33:31.888327 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.888331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.888500 (+ 169us) server_negotiation.cc:407] Connection header received
0807 08:33:31.888634 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.888637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.888676 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.888739 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:31.889001 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.889486 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.890289 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.890545 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.890632 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.891391 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:31.891395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:31.891398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:31.891411 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:31.891427 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:31.891435 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:31.891492 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.891580 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.891633 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 08:33:31.894147 31617 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 08:33:31.897006 31617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 08:33:31.910786 32328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.904563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.66:36021 (local address 127.0.0.1:43052)
0807 08:33:31.904866 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.904910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.905023 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.906602 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.906608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.906629 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:31.906758 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.906765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908164 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.908167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.909398 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.909407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.910220 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.910358 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.910401 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.910513 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.910628 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":37,"thread_start_us":77,"threads_started":1}
I20260807 08:33:31.910815 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.904724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.1.157.66:36021)
0807 08:33:31.905012 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.905023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.905051 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:33:31.906215 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.906223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.906455 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.906599 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:31.906891 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908056 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.909517 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.910100 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910208 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.910426 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.910544 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.910642 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260807 08:33:31.911332 32330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.905247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41959 (local address 127.0.0.1:40248)
0807 08:33:31.905793 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.905806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.905891 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.906807 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.906810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.906827 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:31.906914 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.906920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908374 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.908377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.909905 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.909916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.910980 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.911073 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.911096 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.911141 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.911205 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":294,"thread_start_us":88,"threads_started":1}
I20260807 08:33:31.911535 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.905539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:40547 (local address 127.0.0.1:41040)
0807 08:33:31.906073 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.906085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.906157 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.906944 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.906947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.906966 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:31.907066 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.907072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908397 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.908400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.909905 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.909916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.911151 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.911154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.911250 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.911275 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.911344 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.911411 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":354,"thread_start_us":82,"threads_started":1}
I20260807 08:33:31.911962 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.905933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.1.157.67:40547)
0807 08:33:31.906327 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.906365 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.906388 (+ 23us) server_negotiation.cc:407] Connection header received
0807 08:33:31.906461 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.906466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.906648 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.906803 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:31.907196 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908283 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910035 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.910829 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910933 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.911094 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.911705 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.911801 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":94,"threads_started":1}
I20260807 08:33:31.912147 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.905421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.1.157.65:41959)
0807 08:33:31.906419 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.906430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.906448 (+ 18us) server_negotiation.cc:407] Connection header received
0807 08:33:31.906509 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.906513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.906649 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.906818 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:31.907041 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.908224 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.910070 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.910843 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.911578 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.911856 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.911911 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.911998 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":94,"threads_started":1}
I20260807 08:33:31.914039 32023 tablet_service.cc:1467] Processing CreateTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 (DEFAULT_TABLE table=test-table [id=754ef05cb418441db5011f88367b69e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:31.915030 32023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12792354b8a4516bbf6ad3d5fd6a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:31.914624 31815 tablet_service.cc:1467] Processing CreateTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 (DEFAULT_TABLE table=test-table [id=754ef05cb418441db5011f88367b69e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:31.915660 31815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12792354b8a4516bbf6ad3d5fd6a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:31.915385 32214 tablet_service.cc:1467] Processing CreateTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 (DEFAULT_TABLE table=test-table [id=754ef05cb418441db5011f88367b69e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 08:33:31.916297 32214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12792354b8a4516bbf6ad3d5fd6a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:31.921397 32335 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Bootstrap starting.
I20260807 08:33:31.922796 32336 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Bootstrap starting.
I20260807 08:33:31.923631 32337 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Bootstrap starting.
I20260807 08:33:31.923655 32335 tablet_bootstrap.cc:654] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:31.924507 32335 log.cc:824] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:31.924937 32336 tablet_bootstrap.cc:654] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:31.925298 32337 tablet_bootstrap.cc:654] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:31.925683 32336 log.cc:824] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:31.925973 32337 log.cc:824] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:31.926857 32335 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: No bootstrap required, opened a new log
I20260807 08:33:31.927073 32335 ts_tablet_manager.cc:1397] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 08:33:31.927878 32336 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: No bootstrap required, opened a new log
I20260807 08:33:31.927918 32337 tablet_bootstrap.cc:492] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: No bootstrap required, opened a new log
I20260807 08:33:31.928061 32336 ts_tablet_manager.cc:1397] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:33:31.928084 32337 ts_tablet_manager.cc:1397] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 08:33:31.930996 32337 raft_consensus.cc:348] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.931203 32337 raft_consensus.cc:374] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:31.931274 32337 raft_consensus.cc:729] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06f1d619cbe4a8cb0901462c07a1871, State: Initialized, Role: FOLLOWER
I20260807 08:33:31.931223 32335 raft_consensus.cc:348] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.931483 32335 raft_consensus.cc:374] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:31.931530 32335 raft_consensus.cc:729] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa77a2ebdb964f2a9394bd558451f97f, State: Initialized, Role: FOLLOWER
I20260807 08:33:31.931702 32337 consensus_queue.cc:260] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.931981 32335 consensus_queue.cc:260] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.931980 32336 raft_consensus.cc:348] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.932232 32336 raft_consensus.cc:374] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:31.932296 32336 raft_consensus.cc:729] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee22176403b479a8e2e08c6d9d2158d, State: Initialized, Role: FOLLOWER
I20260807 08:33:31.932400 32310 heartbeater.cc:503] Master 127.1.157.126:46695 was elected leader, sending a full tablet report...
I20260807 08:33:31.932642 32337 ts_tablet_manager.cc:1428] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260807 08:33:31.932720 32336 consensus_queue.cc:260] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.933403 31915 heartbeater.cc:503] Master 127.1.157.126:46695 was elected leader, sending a full tablet report...
I20260807 08:33:31.933640 32336 ts_tablet_manager.cc:1428] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 08:33:31.934963 32335 ts_tablet_manager.cc:1428] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260807 08:33:31.935308 32121 heartbeater.cc:503] Master 127.1.157.126:46695 was elected leader, sending a full tablet report...
W20260807 08:33:31.938704 32122 tablet.cc:2369] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:31.983395 32347 raft_consensus.cc:482] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 08:33:31.983589 32347 raft_consensus.cc:504] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.984668 32347 leader_election.cc:288] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ee22176403b479a8e2e08c6d9d2158d (127.1.157.65:41959), c06f1d619cbe4a8cb0901462c07a1871 (127.1.157.67:40547)
I20260807 08:33:31.988417 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.984866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41959 (local address 127.1.157.66:43619)
0807 08:33:31.985035 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.985053 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.985161 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.985440 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.985443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.985461 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:31.985577 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.985584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.986401 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.986404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.987288 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.987298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.988012 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.988016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.988116 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.988139 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.988204 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.988278 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 08:33:31.988416 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.985040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:43619 (local address 127.1.157.65:41959)
0807 08:33:31.985193 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.985199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.985216 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:33:31.985273 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.985276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.985329 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.985408 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:31.985692 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.986260 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.987416 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.987886 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.988002 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.988055 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.988232 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.988298 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 08:33:31.988616 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.985297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.157.66:54537 (local address 127.1.157.67:40547)
0807 08:33:31.985418 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:31.985422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:31.985583 (+ 161us) server_negotiation.cc:407] Connection header received
0807 08:33:31.985680 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:31.985683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:31.985725 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:31.985786 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 08:33:31.986058 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.986619 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.987628 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.988120 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.988226 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.988273 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:31.988438 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:33:31.988503 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 08:33:31.988624 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:31.985139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.67:40547 (local address 127.1.157.66:54537)
0807 08:33:31.985491 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:31.985502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:31.985592 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:31.985807 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:31.985809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:31.985824 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 08:33:31.985952 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.985958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.986720 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.986723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.987519 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:31.987530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:31.988230 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:31.988234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:31.988331 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:31.988350 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 08:33:31.988415 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 08:33:31.988478 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":135,"threads_started":1}
I20260807 08:33:31.989312 32241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12792354b8a4516bbf6ad3d5fd6a7b9" candidate_uuid: "aa77a2ebdb964f2a9394bd558451f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06f1d619cbe4a8cb0901462c07a1871" is_pre_election: true
I20260807 08:33:31.989328 31845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12792354b8a4516bbf6ad3d5fd6a7b9" candidate_uuid: "aa77a2ebdb964f2a9394bd558451f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee22176403b479a8e2e08c6d9d2158d" is_pre_election: true
I20260807 08:33:31.989578 31845 raft_consensus.cc:2454] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa77a2ebdb964f2a9394bd558451f97f in term 0.
I20260807 08:33:31.989586 32241 raft_consensus.cc:2454] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa77a2ebdb964f2a9394bd558451f97f in term 0.
I20260807 08:33:31.990001 31943 leader_election.cc:302] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa77a2ebdb964f2a9394bd558451f97f, c06f1d619cbe4a8cb0901462c07a1871; no voters:
I20260807 08:33:31.990245 32347 raft_consensus.cc:2781] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 08:33:31.990324 32347 raft_consensus.cc:482] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 08:33:31.990382 32347 raft_consensus.cc:3037] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:31.991403 32347 raft_consensus.cc:504] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.991714 32347 leader_election.cc:288] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [CANDIDATE]: Term 1 election: Requested vote from peers 2ee22176403b479a8e2e08c6d9d2158d (127.1.157.65:41959), c06f1d619cbe4a8cb0901462c07a1871 (127.1.157.67:40547)
I20260807 08:33:31.992087 31845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12792354b8a4516bbf6ad3d5fd6a7b9" candidate_uuid: "aa77a2ebdb964f2a9394bd558451f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee22176403b479a8e2e08c6d9d2158d"
I20260807 08:33:31.992147 32241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12792354b8a4516bbf6ad3d5fd6a7b9" candidate_uuid: "aa77a2ebdb964f2a9394bd558451f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06f1d619cbe4a8cb0901462c07a1871"
I20260807 08:33:31.992205 31845 raft_consensus.cc:3037] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:31.992250 32241 raft_consensus.cc:3037] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:31.993183 31845 raft_consensus.cc:2454] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa77a2ebdb964f2a9394bd558451f97f in term 1.
I20260807 08:33:31.993198 32241 raft_consensus.cc:2454] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa77a2ebdb964f2a9394bd558451f97f in term 1.
I20260807 08:33:31.993461 31943 leader_election.cc:302] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa77a2ebdb964f2a9394bd558451f97f, c06f1d619cbe4a8cb0901462c07a1871; no voters:
I20260807 08:33:31.993639 32347 raft_consensus.cc:2781] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:31.993851 32347 raft_consensus.cc:686] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 1 LEADER]: Becoming Leader. State: Replica: aa77a2ebdb964f2a9394bd558451f97f, State: Running, Role: LEADER
I20260807 08:33:31.994148 32347 consensus_queue.cc:237] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } }
I20260807 08:33:31.997069 31616 catalog_manager.cc:5495] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f reported cstate change: term changed from 0 to 1, leader changed from <none> to aa77a2ebdb964f2a9394bd558451f97f (127.1.157.66). New cstate: current_term: 1 leader_uuid: "aa77a2ebdb964f2a9394bd558451f97f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa77a2ebdb964f2a9394bd558451f97f" member_type: VOTER last_known_addr { host: "127.1.157.66" port: 36021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 } health_report { overall_health: UNKNOWN } } }
W20260807 08:33:32.020709 31916 tablet.cc:2369] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:33:32.029392 32329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:32.026399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.1.157.66:36021)
0807 08:33:32.026523 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:32.026526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:32.026539 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:33:32.026610 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:32.026612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:32.026666 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:32.026747 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:33:32.026943 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:32.027435 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:32.028197 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:32.028404 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:32.028470 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:32.028686 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:33:32.029057 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:33:32.029143 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:32.029176 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 08:33:32.029220 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 08:33:32.034763 32241 raft_consensus.cc:1261] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 1 FOLLOWER]: Refusing update from remote peer aa77a2ebdb964f2a9394bd558451f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:32.034765 31845 raft_consensus.cc:1261] T f12792354b8a4516bbf6ad3d5fd6a7b9 P 2ee22176403b479a8e2e08c6d9d2158d [term 1 FOLLOWER]: Refusing update from remote peer aa77a2ebdb964f2a9394bd558451f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:33:32.035392 32347 consensus_queue.cc:1035] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ee22176403b479a8e2e08c6d9d2158d" member_type: VOTER last_known_addr { host: "127.1.157.65" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:32.035578 32351 consensus_queue.cc:1035] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06f1d619cbe4a8cb0901462c07a1871" member_type: VOTER last_known_addr { host: "127.1.157.67" port: 40547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:33:32.050976 31616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:32.051143 31616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 08:33:32.053412 31616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88fc73bbc5a54a39a58123eeb5f3d670: Sending DeleteTablet for 3 replicas of tablet f12792354b8a4516bbf6ad3d5fd6a7b9
I20260807 08:33:32.054077 32023 tablet_service.cc:1514] Processing DeleteTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:32 UTC) from {username='slave'} at 127.0.0.1:43052
I20260807 08:33:32.054219 32214 tablet_service.cc:1514] Processing DeleteTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:32 UTC) from {username='slave'} at 127.0.0.1:41040
I20260807 08:33:32.054301 31815 tablet_service.cc:1514] Processing DeleteTablet for tablet f12792354b8a4516bbf6ad3d5fd6a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 08:33:32 UTC) from {username='slave'} at 127.0.0.1:40248
I20260807 08:33:32.054668 32377 tablet_replica.cc:330] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: stopping tablet replica
I20260807 08:33:32.055032 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 31705
I20260807 08:33:32.055027 32377 raft_consensus.cc:2229] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 08:33:32.061950 32377 raft_consensus.cc:2258] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:32.062062 32375 tablet_replica.cc:330] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: stopping tablet replica
I20260807 08:33:32.062372 32375 raft_consensus.cc:2229] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 1 LEADER]: Raft consensus shutting down.
I20260807 08:33:32.062737 32375 raft_consensus.cc:2258] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 08:33:32.063719 32377 ts_tablet_manager.cc:1905] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:32.064023 32375 ts_tablet_manager.cc:1905] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 08:33:32.066506 32377 ts_tablet_manager.cc:1918] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:32.066660 32377 log.cc:1196] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-2/wal/wals/f12792354b8a4516bbf6ad3d5fd6a7b9
I20260807 08:33:32.066715 32375 ts_tablet_manager.cc:1918] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 08:33:32.066886 32375 log.cc:1196] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Deleting WAL directory at /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786091364559477-1653-0/minicluster-data/ts-1/wal/wals/f12792354b8a4516bbf6ad3d5fd6a7b9
I20260807 08:33:32.067054 32377 ts_tablet_manager.cc:1939] T f12792354b8a4516bbf6ad3d5fd6a7b9 P c06f1d619cbe4a8cb0901462c07a1871: Deleting consensus metadata
I20260807 08:33:32.067241 32375 ts_tablet_manager.cc:1939] T f12792354b8a4516bbf6ad3d5fd6a7b9 P aa77a2ebdb964f2a9394bd558451f97f: Deleting consensus metadata
I20260807 08:33:32.068249 31592 catalog_manager.cc:4841] TS c06f1d619cbe4a8cb0901462c07a1871 (127.1.157.67:40547): tablet f12792354b8a4516bbf6ad3d5fd6a7b9 (table test-table [id=754ef05cb418441db5011f88367b69e2]) successfully deleted
I20260807 08:33:32.068442 31592 catalog_manager.cc:4841] TS aa77a2ebdb964f2a9394bd558451f97f (127.1.157.66:36021): tablet f12792354b8a4516bbf6ad3d5fd6a7b9 (table test-table [id=754ef05cb418441db5011f88367b69e2]) successfully deleted
W20260807 08:33:32.069507 31592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.157.65:41959 (error 108)
I20260807 08:33:32.069679 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 31919
I20260807 08:33:32.079246 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:32.078901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.65:41959 (local address 127.0.0.1:40254)
0807 08:33:32.079067 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:32.079152 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.65:41959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 08:33:32.079420 31592 catalog_manager.cc:4584] TS 2ee22176403b479a8e2e08c6d9d2158d (127.1.157.65:41959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f12792354b8a4516bbf6ad3d5fd6a7b9: Network error: Client connection negotiation failed: client connection to 127.1.157.65:41959: connect: Connection refused (error 111)
I20260807 08:33:32.083058 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 32124
I20260807 08:33:32.097621 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 31570
2026-08-07T08:33:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1036 ms)
[----------] 6 tests from EncryptionPolicyTest (10059 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:32 dist-test-slave-6zbq krb5kdc[32389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:32 dist-test-slave-6zbq krb5kdc[32389](info): set up 2 sockets
Aug 07 08:33:32 dist-test-slave-6zbq krb5kdc[32389](info): commencing operation
krb5kdc: starting...
W20260807 08:33:33.222620 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:33.342979 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34805
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:33.448457 32410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:33.448668 32410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:33.448709 32410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:33.452553 32410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:33.452601 32410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:33.452621 32410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:33.452651 32410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:33.452677 32410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:33.457597 32410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:33.458473 32410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:33.459375 32410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:33:33.467041 32410 server_base.cc:1034] running on GCE node
W20260807 08:33:33.466990 32419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:33.467075 32421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:33.466986 32425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:33.467654 32410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:33.468606 32410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:33.469760 32410 hybrid_clock.cc:648] HybridClock initialized: now 1786091613469747 us; error 23 us; skew 500 ppm
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091613, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:33.472194 32410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:33.472934 32410 webserver.cc:460] Webserver started at http://127.0.0.1:35863/ using document root <none> and password file <none>
I20260807 08:33:33.473374 32410 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:33.473425 32410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:33.473572 32410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:33.475397 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "8419a60e20514e2e8f7002d15e3b14dd"
format_stamp: "Formatted at 2026-08-07 08:33:33 on dist-test-slave-6zbq"
server_key: "044a44b332742950784eed744ca4ddd8"
server_key_iv: "13c88503a4af8425405ce5eb670f29ea"
server_key_version: "encryptionkey@0"
I20260807 08:33:33.475876 32410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "8419a60e20514e2e8f7002d15e3b14dd"
format_stamp: "Formatted at 2026-08-07 08:33:33 on dist-test-slave-6zbq"
server_key: "044a44b332742950784eed744ca4ddd8"
server_key_iv: "13c88503a4af8425405ce5eb670f29ea"
server_key_version: "encryptionkey@0"
I20260807 08:33:33.479153 32410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:33:33.481235 32437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:33.482270 32410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:33.482393 32410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "8419a60e20514e2e8f7002d15e3b14dd"
format_stamp: "Formatted at 2026-08-07 08:33:33 on dist-test-slave-6zbq"
server_key: "044a44b332742950784eed744ca4ddd8"
server_key_iv: "13c88503a4af8425405ce5eb670f29ea"
server_key_version: "encryptionkey@0"
I20260807 08:33:33.482486 32410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:33.495216 32410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:33.497522 32410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:33.497673 32410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:33.504083 32410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34805
I20260807 08:33:33.504108 32512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34805 every 8 connection(s)
I20260807 08:33:33.504875 32410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:33.507383 32513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:33.507668 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 32410
I20260807 08:33:33.507789 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:33.508013 1653 external_mini_cluster.cc:1428] Setting key 2e606e99185e037a5264c75e668ef7f2
I20260807 08:33:33.514153 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: Bootstrap starting.
I20260807 08:33:33.516412 32513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: Neither blocks nor log segments found. Creating new log.
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:33:33.517124 32513 log.cc:824] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:33.519222 32513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: No bootstrap required, opened a new log
I20260807 08:33:33.522112 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:33.509222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35115 (local address 127.0.0.1:34805)
0807 08:33:33.509669 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:33.509688 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:33.509728 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:33:33.510599 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:33.510624 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:33.511030 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:33.511391 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:33.511995 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:33.513451 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:33.514258 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:33.514931 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:33.515099 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:33.517312 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:33.517319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:33.517334 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:33.517362 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:33.519210 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:33.519675 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:33.519680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:33.519686 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:33.519825 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:33.520100 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:33.520103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:33.520105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:33.520563 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:33.520760 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:33.520964 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 08:33:33.521215 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":111,"threads_started":1}
I20260807 08:33:33.523460 32513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260807 08:33:33.523666 32513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:33.523746 32513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8419a60e20514e2e8f7002d15e3b14dd, State: Initialized, Role: FOLLOWER
I20260807 08:33:33.524156 32513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260807 08:33:33.524297 32513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:33.524366 32513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:33.524466 32513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:33.524842 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:33.525346 32513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260807 08:33:33.525652 32513 leader_election.cc:302] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8419a60e20514e2e8f7002d15e3b14dd; no voters:
I20260807 08:33:33.525952 32513 leader_election.cc:288] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:33.526021 32523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:33.526265 32523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [term 1 LEADER]: Becoming Leader. State: Replica: 8419a60e20514e2e8f7002d15e3b14dd, State: Running, Role: LEADER
I20260807 08:33:33.526638 32523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260807 08:33:33.527177 32513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:33.527741 32523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8419a60e20514e2e8f7002d15e3b14dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } } }
I20260807 08:33:33.527892 32523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:33.528200 32526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8419a60e20514e2e8f7002d15e3b14dd. Latest consensus state: current_term: 1 leader_uuid: "8419a60e20514e2e8f7002d15e3b14dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8419a60e20514e2e8f7002d15e3b14dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } } }
I20260807 08:33:33.528363 32526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:33.531643 32550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:33.531715 32550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:33.531804 32541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:33.532523 32541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:33.536427 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:33.526726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56233 (local address 127.0.0.1:34805)
0807 08:33:33.526817 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:33.526821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:33.528331 (+ 1510us) server_negotiation.cc:407] Connection header received
0807 08:33:33.528501 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:33.528504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:33.528545 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:33.528589 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:33.529389 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:33.530021 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:33.530990 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:33.531314 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:33.531386 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:33.533020 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:33.533024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:33.533027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:33.533052 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:33.534608 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:33.535122 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:33.535125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:33.535127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:33.535211 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:33.535562 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:33.535565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:33.535567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:33.535781 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:33.535922 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:33.536193 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 08:33:33.536285 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:33:33.538733 32541 catalog_manager.cc:1347] Generated new cluster ID: 9730c4496ef149e4b03e5255e5d02e74
I20260807 08:33:33.538793 32541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:33.556332 32541 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:33.557469 32541 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:33.570712 32541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8419a60e20514e2e8f7002d15e3b14dd: Generated new TSK 0
I20260807 08:33:33.571379 32541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:33.575824 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 32410
W20260807 08:33:33.588680 32383 connection.cc:535] client connection to 127.0.0.1:34805 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34805): Connection reset by peer (error 104)
2026-08-07T08:33:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32563](info): set up 2 sockets
Aug 07 08:33:33 dist-test-slave-6zbq krb5kdc[32563](info): commencing operation
krb5kdc: starting...
W20260807 08:33:34.704941 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:34 dist-test-slave-6zbq krb5kdc[32563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:34.814955 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:34.919930 32579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:34.920148 32579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:34.920185 32579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:34.923970 32579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:34.924015 32579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:34.924032 32579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:34.924046 32579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:34.924065 32579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:34.928893 32579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:34.929687 32579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:34.930538 32579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:34.938100 32586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:34.938174 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:34.938158 32579 server_base.cc:1034] running on GCE node
W20260807 08:33:34.938108 32585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:34.938817 32579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:34.939771 32579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:34.940975 32579 hybrid_clock.cc:648] HybridClock initialized: now 1786091614940960 us; error 25 us; skew 500 ppm
Aug 07 08:33:34 dist-test-slave-6zbq krb5kdc[32563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091614, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:34.943521 32579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:34.944344 32579 webserver.cc:460] Webserver started at http://127.0.0.1:42405/ using document root <none> and password file <none>
I20260807 08:33:34.944877 32579 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:34.944942 32579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:34.945129 32579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:34.947057 32579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "2d066237c2254eb48ca8e1cdb1183f63"
format_stamp: "Formatted at 2026-08-07 08:33:34 on dist-test-slave-6zbq"
server_key: "73efb49c179cad8406371da88efbe9ff"
server_key_iv: "172c546c83de20cea9235f3c2c6f67d4"
server_key_version: "encryptionkey@0"
I20260807 08:33:34.947572 32579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "2d066237c2254eb48ca8e1cdb1183f63"
format_stamp: "Formatted at 2026-08-07 08:33:34 on dist-test-slave-6zbq"
server_key: "73efb49c179cad8406371da88efbe9ff"
server_key_iv: "172c546c83de20cea9235f3c2c6f67d4"
server_key_version: "encryptionkey@0"
I20260807 08:33:34.950946 32579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:33:34.953227 32599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:34.954319 32579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:34.954463 32579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "2d066237c2254eb48ca8e1cdb1183f63"
format_stamp: "Formatted at 2026-08-07 08:33:34 on dist-test-slave-6zbq"
server_key: "73efb49c179cad8406371da88efbe9ff"
server_key_iv: "172c546c83de20cea9235f3c2c6f67d4"
server_key_version: "encryptionkey@0"
I20260807 08:33:34.954553 32579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:34.965787 32579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:34.968295 32579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:34.968454 32579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:34.975059 32579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46459
I20260807 08:33:34.975073 32682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46459 every 8 connection(s)
I20260807 08:33:34.975909 32579 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:34.978463 32684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:34.979682 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 32579
I20260807 08:33:34.979822 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:34.980130 1653 external_mini_cluster.cc:1428] Setting key 59c59eb63db687ae2c1d3782a4d1c3d5
I20260807 08:33:34.984342 32684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63: Bootstrap starting.
I20260807 08:33:34.986831 32684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:34.987571 32684 log.cc:824] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:34.988494 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:34.981347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42375 (local address 127.0.0.1:46459)
0807 08:33:34.981726 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:34.981740 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:34.981773 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:34.982419 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:34.982437 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:34.982751 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:34.983007 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:34.983310 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:34.984213 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:34.985374 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:34.985825 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:34.985945 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:34.986803 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:34.986808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:34.986820 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:34.987189 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:34.987251 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:34.987273 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:34.987371 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:34.987556 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:33:34.987676 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
I20260807 08:33:34.989382 32684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63: No bootstrap required, opened a new log
I20260807 08:33:34.991039 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:34.993199 32684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260807 08:33:34.993469 32684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:34.993542 32684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d066237c2254eb48ca8e1cdb1183f63, State: Initialized, Role: FOLLOWER
I20260807 08:33:34.993953 32684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260807 08:33:34.994077 32684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:34.994140 32684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:34.994232 32684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:34.995035 32684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260807 08:33:34.995333 32684 leader_election.cc:302] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d066237c2254eb48ca8e1cdb1183f63; no voters:
I20260807 08:33:34.995584 32684 leader_election.cc:288] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:34.995682 32700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:34.996209 32700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [term 1 LEADER]: Becoming Leader. State: Replica: 2d066237c2254eb48ca8e1cdb1183f63, State: Running, Role: LEADER
I20260807 08:33:34.996500 32700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } }
I20260807 08:33:34.996831 32684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:34.996984 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:34.993129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35185 (local address 127.0.0.1:46459)
0807 08:33:34.993246 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:34.993250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:34.993378 (+ 128us) server_negotiation.cc:407] Connection header received
0807 08:33:34.993492 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:34.993495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:34.993536 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:34.993605 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:34.993881 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:34.994499 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:34.995264 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:34.995557 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:34.995687 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:34.996494 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:34.996498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:34.996500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:34.996513 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:34.996539 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:34.996552 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:34.996638 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:34.996843 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 08:33:34.996876 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:33:34.997773 32700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d066237c2254eb48ca8e1cdb1183f63. Latest consensus state: current_term: 1 leader_uuid: "2d066237c2254eb48ca8e1cdb1183f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260807 08:33:34.997956 32700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:34.998171 32701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d066237c2254eb48ca8e1cdb1183f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d066237c2254eb48ca8e1cdb1183f63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46459 } } }
I20260807 08:33:34.998271 32701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:34.998332 32714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:35.001222 32714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:35.006091 32714 catalog_manager.cc:1347] Generated new cluster ID: 2289b9b649954c2394d4f7e78f4ca03e
I20260807 08:33:35.006165 32714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:35.022631 32714 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:35.023749 32714 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:35.031118 32714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d066237c2254eb48ca8e1cdb1183f63: Generated new TSK 0
I20260807 08:33:35.031662 32714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:35.044100 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 32579
W20260807 08:33:35.056375 32558 connection.cc:535] client connection to 127.0.0.1:46459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46459): Connection reset by peer (error 104)
2026-08-07T08:33:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:35 dist-test-slave-6zbq krb5kdc[32730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:35 dist-test-slave-6zbq krb5kdc[32730](info): set up 2 sockets
Aug 07 08:33:35 dist-test-slave-6zbq krb5kdc[32730](info): commencing operation
krb5kdc: starting...
W20260807 08:33:36.156179 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:36 dist-test-slave-6zbq krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:36.265429 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:36.370741 32746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:36.370963 32746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:36.371018 32746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:36.374840 32746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:36.374890 32746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:36.374915 32746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:36.374936 32746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:36.374961 32746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:36.379865 32746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:36.380729 32746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:36.381614 32746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:36.389173 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:36.389150 32753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:36.389283 32746 server_base.cc:1034] running on GCE node
W20260807 08:33:36.389165 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:36.389962 32746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:36.390959 32746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:36.392113 32746 hybrid_clock.cc:648] HybridClock initialized: now 1786091616392096 us; error 30 us; skew 500 ppm
Aug 07 08:33:36 dist-test-slave-6zbq krb5kdc[32730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:36.394551 32746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:36.395313 32746 webserver.cc:460] Webserver started at http://127.0.0.1:45005/ using document root <none> and password file <none>
I20260807 08:33:36.395787 32746 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:36.395852 32746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:36.396035 32746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:36.397828 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "4dd559274b89484ba05eb39c6bcd8479"
format_stamp: "Formatted at 2026-08-07 08:33:36 on dist-test-slave-6zbq"
server_key: "1820090b85d038a498c755be06c5d12f"
server_key_iv: "b3ba7715749726c09aca2b841a05bae5"
server_key_version: "encryptionkey@0"
I20260807 08:33:36.398327 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "4dd559274b89484ba05eb39c6bcd8479"
format_stamp: "Formatted at 2026-08-07 08:33:36 on dist-test-slave-6zbq"
server_key: "1820090b85d038a498c755be06c5d12f"
server_key_iv: "b3ba7715749726c09aca2b841a05bae5"
server_key_version: "encryptionkey@0"
I20260807 08:33:36.401590 32746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260807 08:33:36.403761 32763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:36.404798 32746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:33:36.404906 32746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "4dd559274b89484ba05eb39c6bcd8479"
format_stamp: "Formatted at 2026-08-07 08:33:36 on dist-test-slave-6zbq"
server_key: "1820090b85d038a498c755be06c5d12f"
server_key_iv: "b3ba7715749726c09aca2b841a05bae5"
server_key_version: "encryptionkey@0"
I20260807 08:33:36.404992 32746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:36.409097 32746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:36.411552 32746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:36.411713 32746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:36.417829 32746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36967
I20260807 08:33:36.417841 383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36967 every 8 connection(s)
I20260807 08:33:36.418622 32746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:36.420193 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 32746
I20260807 08:33:36.420334 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:36.420595 1653 external_mini_cluster.cc:1428] Setting key 320a2321affa128eb2ed7f942ceffb05
I20260807 08:33:36.421413 384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:36.425057 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:36.421771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51711 (local address 127.0.0.1:36967)
0807 08:33:36.422122 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:36.422136 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:36.422168 (+ 32us) server_negotiation.cc:407] Connection header received
0807 08:33:36.422762 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:36.422777 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:36.423021 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:36.423265 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:36.423999 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:36.424003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:36.424014 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:36.424023 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:36.424069 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:36.424086 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:36.424189 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:36.424301 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 08:33:36.424421 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
I20260807 08:33:36.427244 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479: Bootstrap starting.
I20260807 08:33:36.427546 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:36.429332 384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:36.430164 384 log.cc:824] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:36.430799 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:36.429354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55781 (local address 127.0.0.1:36967)
0807 08:33:36.429452 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:36.429456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:36.429659 (+ 203us) server_negotiation.cc:407] Connection header received
0807 08:33:36.429762 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:36.429764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:36.429790 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:36.429848 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:36.430466 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:36.430470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:36.430473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:36.430485 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:36.430510 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:36.430523 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:36.430606 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:36.430692 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 08:33:36.430719 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:33:36.431952 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479: No bootstrap required, opened a new log
I20260807 08:33:36.435139 384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 08:33:36.435335 384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:36.435393 384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd559274b89484ba05eb39c6bcd8479, State: Initialized, Role: FOLLOWER
I20260807 08:33:36.435750 384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 08:33:36.435868 384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:36.435930 384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:36.436023 384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:36.436764 384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 08:33:36.437058 384 leader_election.cc:302] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd559274b89484ba05eb39c6bcd8479; no voters:
I20260807 08:33:36.437318 384 leader_election.cc:288] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:36.437404 399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:36.437628 399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd559274b89484ba05eb39c6bcd8479, State: Running, Role: LEADER
I20260807 08:33:36.437937 399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } }
I20260807 08:33:36.438380 384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:36.439394 401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dd559274b89484ba05eb39c6bcd8479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260807 08:33:36.439523 401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:36.439862 404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dd559274b89484ba05eb39c6bcd8479. Latest consensus state: current_term: 1 leader_uuid: "4dd559274b89484ba05eb39c6bcd8479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd559274b89484ba05eb39c6bcd8479" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36967 } } }
I20260807 08:33:36.440083 414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:36.440434 404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:36.443094 414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:36.447531 414 catalog_manager.cc:1347] Generated new cluster ID: ca990907bfce42bf9c12c62617ecf048
I20260807 08:33:36.447589 414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:36.461279 414 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:36.461570 414 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:36.467502 414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dd559274b89484ba05eb39c6bcd8479: Generated new TSK 0
I20260807 08:33:36.468152 414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:36.475097 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 32746
2026-08-07T08:33:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:36 dist-test-slave-6zbq krb5kdc[434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:36 dist-test-slave-6zbq krb5kdc[434](info): set up 2 sockets
Aug 07 08:33:36 dist-test-slave-6zbq krb5kdc[434](info): commencing operation
krb5kdc: starting...
W20260807 08:33:37.598834 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:37 dist-test-slave-6zbq krb5kdc[434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:37.708217 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:37.811442 453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:37.811656 453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:37.811700 453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:37.815442 453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:37.815487 453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:37.815506 453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:37.815521 453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:37.815541 453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:37.820358 453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:37.821183 453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:37.822022 453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:37.829576 463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:37.829586 465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:37.829689 462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:37.829663 453 server_base.cc:1034] running on GCE node
I20260807 08:33:37.830315 453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:37.831293 453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:37.832446 453 hybrid_clock.cc:648] HybridClock initialized: now 1786091617832427 us; error 32 us; skew 500 ppm
Aug 07 08:33:37 dist-test-slave-6zbq krb5kdc[434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:37.834964 453 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:37.835781 453 webserver.cc:460] Webserver started at http://127.0.0.1:43011/ using document root <none> and password file <none>
I20260807 08:33:37.836344 453 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:37.836431 453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:37.836630 453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:37.838865 453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "c6de53609482480c9524d0220b4b6406"
format_stamp: "Formatted at 2026-08-07 08:33:37 on dist-test-slave-6zbq"
server_key: "cc4b9219beed0f153bc64f66ac695650"
server_key_iv: "3f4784395beda8e2a5a0f0b496f1202e"
server_key_version: "encryptionkey@0"
I20260807 08:33:37.839375 453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "c6de53609482480c9524d0220b4b6406"
format_stamp: "Formatted at 2026-08-07 08:33:37 on dist-test-slave-6zbq"
server_key: "cc4b9219beed0f153bc64f66ac695650"
server_key_iv: "3f4784395beda8e2a5a0f0b496f1202e"
server_key_version: "encryptionkey@0"
I20260807 08:33:37.842761 453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:37.844832 474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:37.845932 453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:37.846040 453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c6de53609482480c9524d0220b4b6406"
format_stamp: "Formatted at 2026-08-07 08:33:37 on dist-test-slave-6zbq"
server_key: "cc4b9219beed0f153bc64f66ac695650"
server_key_iv: "3f4784395beda8e2a5a0f0b496f1202e"
server_key_version: "encryptionkey@0"
I20260807 08:33:37.846127 453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:37.879032 453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:37.881774 453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:37.881942 453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:37.888356 453 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38099
I20260807 08:33:37.888372 554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38099 every 8 connection(s)
I20260807 08:33:37.889231 453 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:37.891718 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:37.893074 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 453
I20260807 08:33:37.893186 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:37.893428 1653 external_mini_cluster.cc:1428] Setting key e661b83394c7253f11ec654c86437c7a
I20260807 08:33:37.897677 556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: Bootstrap starting.
I20260807 08:33:37.900027 556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:37.900795 556 log.cc:824] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:33:37 dist-test-slave-6zbq krb5kdc[434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:33:37.902585 556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: No bootstrap required, opened a new log
I20260807 08:33:37.905918 556 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260807 08:33:37.906175 556 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:37.906245 556 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6de53609482480c9524d0220b4b6406, State: Initialized, Role: FOLLOWER
I20260807 08:33:37.906675 556 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260807 08:33:37.906806 556 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:37.906870 556 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:37.906878 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:37.894687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38211 (local address 127.0.0.1:38099)
0807 08:33:37.895072 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:37.895090 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:37.895130 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:33:37.895919 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:37.895941 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:37.896337 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:37.896669 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:37.897312 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:37.898729 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:37.899557 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:37.900239 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:37.900393 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:37.902749 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:37.902754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:37.902766 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:37.902791 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:37.904256 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:37.904647 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:37.904653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:37.904657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:37.904765 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:37.905018 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:37.905021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:37.905023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:37.905385 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:37.905575 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:37.905847 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 08:33:37.906106 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":89,"threads_started":1}
I20260807 08:33:37.906968 556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:37.907938 556 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260807 08:33:37.908242 556 leader_election.cc:302] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6de53609482480c9524d0220b4b6406; no voters:
I20260807 08:33:37.908496 556 leader_election.cc:288] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:37.908594 564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:37.908820 564 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [term 1 LEADER]: Becoming Leader. State: Replica: c6de53609482480c9524d0220b4b6406, State: Running, Role: LEADER
I20260807 08:33:37.909130 564 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260807 08:33:37.909632 556 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:37.909657 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:37.910724 567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6de53609482480c9524d0220b4b6406. Latest consensus state: current_term: 1 leader_uuid: "c6de53609482480c9524d0220b4b6406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260807 08:33:37.910846 567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:37.911188 566 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6de53609482480c9524d0220b4b6406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6de53609482480c9524d0220b4b6406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260807 08:33:37.911346 566 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:37.914789 596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:37.914867 596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:37.914923 581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:37.915730 581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:37.919464 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:37.911920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48379 (local address 127.0.0.1:38099)
0807 08:33:37.912029 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:37.912033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:37.912703 (+ 670us) server_negotiation.cc:407] Connection header received
0807 08:33:37.912814 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:37.912816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:37.912849 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:37.912918 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:37.913524 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:37.914006 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:37.914816 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:37.915146 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:37.915245 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:37.916425 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:37.916429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:37.916432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:37.916457 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:37.917754 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:37.918236 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:37.918240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:37.918241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:37.918330 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:37.918629 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:37.918632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:37.918634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:37.918835 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:37.918955 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:37.919213 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 08:33:37.919325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:33:37.923770 581 catalog_manager.cc:1347] Generated new cluster ID: 292ea8daef1f46fea0ada1de0106efe8
I20260807 08:33:37.923821 581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:37.929505 581 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:37.930266 581 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:37.937538 581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6de53609482480c9524d0220b4b6406: Generated new TSK 0
I20260807 08:33:37.938060 581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:37.947183 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 453
2026-08-07T08:33:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:37 dist-test-slave-6zbq krb5kdc[603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:38 dist-test-slave-6zbq krb5kdc[603](info): set up 2 sockets
Aug 07 08:33:38 dist-test-slave-6zbq krb5kdc[603](info): commencing operation
krb5kdc: starting...
W20260807 08:33:39.058081 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:39 dist-test-slave-6zbq krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:39.166613 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:39.270802 622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:39.271025 622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:39.271083 622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:39.274930 622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:39.274979 622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:39.275005 622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:39.275028 622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:39.275058 622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:39.280290 622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:39.281461 622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:39.282725 622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:39.293963 632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:39.294035 622 server_base.cc:1034] running on GCE node
W20260807 08:33:39.293959 630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:39.293988 629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:39.294675 622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:39.295678 622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:39.296829 622 hybrid_clock.cc:648] HybridClock initialized: now 1786091619296812 us; error 40 us; skew 500 ppm
Aug 07 08:33:39 dist-test-slave-6zbq krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:39.302423 622 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:39.303247 622 webserver.cc:460] Webserver started at http://127.0.0.1:41829/ using document root <none> and password file <none>
I20260807 08:33:39.303735 622 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:39.303788 622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:39.303951 622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:39.305873 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "c902b4f96fe14862a1f6a397d5541081"
format_stamp: "Formatted at 2026-08-07 08:33:39 on dist-test-slave-6zbq"
server_key: "411ad0c0f3a38bab752695a57663657d"
server_key_iv: "dac4df030f20ee5eded6936cbc4e511b"
server_key_version: "encryptionkey@0"
I20260807 08:33:39.306401 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "c902b4f96fe14862a1f6a397d5541081"
format_stamp: "Formatted at 2026-08-07 08:33:39 on dist-test-slave-6zbq"
server_key: "411ad0c0f3a38bab752695a57663657d"
server_key_iv: "dac4df030f20ee5eded6936cbc4e511b"
server_key_version: "encryptionkey@0"
I20260807 08:33:39.309798 622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:33:39.312063 643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:39.313160 622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 08:33:39.313275 622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c902b4f96fe14862a1f6a397d5541081"
format_stamp: "Formatted at 2026-08-07 08:33:39 on dist-test-slave-6zbq"
server_key: "411ad0c0f3a38bab752695a57663657d"
server_key_iv: "dac4df030f20ee5eded6936cbc4e511b"
server_key_version: "encryptionkey@0"
I20260807 08:33:39.313369 622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:39.322731 622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:39.325145 622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:39.325291 622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:39.331691 622 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35115
I20260807 08:33:39.331708 727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35115 every 8 connection(s)
I20260807 08:33:39.332517 622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:39.335047 729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:39.340090 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: Bootstrap starting.
I20260807 08:33:39.341686 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 622
I20260807 08:33:39.341822 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:39.342058 1653 external_mini_cluster.cc:1428] Setting key 6b30faead989a1815f0cbf8f5c494f57
I20260807 08:33:39.342522 729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:39.343336 729 log.cc:824] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:39.345324 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: No bootstrap required, opened a new log
I20260807 08:33:39.349232 729 raft_consensus.cc:348] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } }
I20260807 08:33:39.349478 729 raft_consensus.cc:374] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:39.349545 729 raft_consensus.cc:729] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c902b4f96fe14862a1f6a397d5541081, State: Initialized, Role: FOLLOWER
I20260807 08:33:39.349990 729 consensus_queue.cc:260] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } }
I20260807 08:33:39.350220 729 raft_consensus.cc:388] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:39.350298 729 raft_consensus.cc:482] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:39.350415 729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:39.351190 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:39.343043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32951 (local address 127.0.0.1:35115)
0807 08:33:39.343495 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:39.343513 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:39.343553 (+ 40us) server_negotiation.cc:407] Connection header received
0807 08:33:39.344284 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:39.344305 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:39.344695 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:39.345026 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:39.345417 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:39.346885 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:39.347761 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:39.348474 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:39.348632 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:39.349287 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:39.349292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:39.349307 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:39.349769 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:39.349833 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:39.349861 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:39.349942 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:39.350069 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 08:33:39.350264 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":128,"threads_started":1}
I20260807 08:33:39.351419 729 raft_consensus.cc:504] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } }
I20260807 08:33:39.351724 729 leader_election.cc:302] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c902b4f96fe14862a1f6a397d5541081; no voters:
I20260807 08:33:39.351987 729 leader_election.cc:288] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:39.352077 736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:39.352310 736 raft_consensus.cc:686] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [term 1 LEADER]: Becoming Leader. State: Replica: c902b4f96fe14862a1f6a397d5541081, State: Running, Role: LEADER
I20260807 08:33:39.352643 736 consensus_queue.cc:237] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } }
I20260807 08:33:39.353104 729 sys_catalog.cc:564] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:39.353931 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:39.354161 739 sys_catalog.cc:455] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c902b4f96fe14862a1f6a397d5541081. Latest consensus state: current_term: 1 leader_uuid: "c902b4f96fe14862a1f6a397d5541081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } } }
I20260807 08:33:39.354261 739 sys_catalog.cc:458] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:39.354486 737 sys_catalog.cc:455] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c902b4f96fe14862a1f6a397d5541081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c902b4f96fe14862a1f6a397d5541081" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35115 } } }
I20260807 08:33:39.354558 737 sys_catalog.cc:458] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:39.358016 759 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:39.358093 759 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:39.358160 750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:39.358960 750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:39.360597 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:39.356393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59945 (local address 127.0.0.1:35115)
0807 08:33:39.356508 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:39.356512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:39.356641 (+ 129us) server_negotiation.cc:407] Connection header received
0807 08:33:39.356778 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:39.356780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:39.356835 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:39.356909 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:39.357170 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:39.357927 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:39.358978 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:39.359308 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:39.359415 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:39.360197 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:39.360201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:39.360203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:39.360216 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:39.360243 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:39.360256 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:39.360331 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:39.360423 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 08:33:39.360472 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:33:39.363924 750 catalog_manager.cc:1347] Generated new cluster ID: 1610c38e78e640bdaf9872c6035e0e5f
I20260807 08:33:39.363982 750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:39.376654 750 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:39.377540 750 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:39.391582 750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c902b4f96fe14862a1f6a397d5541081: Generated new TSK 0
I20260807 08:33:39.392190 750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:39.401563 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 622
2026-08-07T08:33:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:39 dist-test-slave-6zbq krb5kdc[777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:39 dist-test-slave-6zbq krb5kdc[777](info): set up 2 sockets
Aug 07 08:33:39 dist-test-slave-6zbq krb5kdc[777](info): commencing operation
krb5kdc: starting...
W20260807 08:33:40.526587 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:40 dist-test-slave-6zbq krb5kdc[777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:40.642611 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:40.751425 794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:40.751679 794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:40.751720 794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:40.755734 794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:40.755795 794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:40.755815 794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:40.755831 794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:40.755858 794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:40.761000 794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:40.761951 794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:40.762931 794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:40.770677 803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:40.770684 806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:40.770694 802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:40.771063 794 server_base.cc:1034] running on GCE node
I20260807 08:33:40.771591 794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:40.772567 794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:40.773746 794 hybrid_clock.cc:648] HybridClock initialized: now 1786091620773732 us; error 29 us; skew 500 ppm
Aug 07 08:33:40 dist-test-slave-6zbq krb5kdc[777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:40.776293 794 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:40.777129 794 webserver.cc:460] Webserver started at http://127.0.0.1:38887/ using document root <none> and password file <none>
I20260807 08:33:40.777640 794 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:40.777704 794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:40.777881 794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:40.779863 794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "7cf3ddf8acd1465cb7317b00e8083649"
format_stamp: "Formatted at 2026-08-07 08:33:40 on dist-test-slave-6zbq"
server_key: "9aa51b9463e6948f39f34e725cf1ef3b"
server_key_iv: "b8c47f0765ab513044ec7f6efa9167dc"
server_key_version: "encryptionkey@0"
I20260807 08:33:40.780388 794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "7cf3ddf8acd1465cb7317b00e8083649"
format_stamp: "Formatted at 2026-08-07 08:33:40 on dist-test-slave-6zbq"
server_key: "9aa51b9463e6948f39f34e725cf1ef3b"
server_key_iv: "b8c47f0765ab513044ec7f6efa9167dc"
server_key_version: "encryptionkey@0"
I20260807 08:33:40.783885 794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:33:40.786077 816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:40.787221 794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:40.787335 794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "7cf3ddf8acd1465cb7317b00e8083649"
format_stamp: "Formatted at 2026-08-07 08:33:40 on dist-test-slave-6zbq"
server_key: "9aa51b9463e6948f39f34e725cf1ef3b"
server_key_iv: "b8c47f0765ab513044ec7f6efa9167dc"
server_key_version: "encryptionkey@0"
I20260807 08:33:40.787426 794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:40.791705 794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:40.794273 794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:40.794477 794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:40.800993 794 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38549
I20260807 08:33:40.801000 884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38549 every 8 connection(s)
I20260807 08:33:40.801859 794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:40.804536 885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:40.808007 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 794
I20260807 08:33:40.808151 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:40.808436 1653 external_mini_cluster.cc:1428] Setting key b08f31be49ccbea513d9645876dbc511
I20260807 08:33:40.810217 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649: Bootstrap starting.
I20260807 08:33:40.812397 885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:40.813163 885 log.cc:824] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:40.813344 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:40.809775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49083 (local address 127.0.0.1:38549)
0807 08:33:40.810234 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:40.810248 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:40.810282 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:33:40.810952 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:40.810968 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:40.811232 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:40.811537 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:40.812238 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:40.812243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:40.812254 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:40.812264 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:40.812315 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:40.812333 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:40.812423 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:40.812541 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 08:33:40.812687 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":121,"threads_started":1}
I20260807 08:33:40.815164 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649: No bootstrap required, opened a new log
I20260807 08:33:40.815833 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:40.818835 885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } }
I20260807 08:33:40.819015 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:40.817599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38391 (local address 127.0.0.1:38549)
0807 08:33:40.817692 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:40.817697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:40.817871 (+ 174us) server_negotiation.cc:407] Connection header received
0807 08:33:40.817967 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:40.817970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:40.817998 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:40.818054 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:40.818694 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:40.818697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:40.818700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:40.818711 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:40.818735 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:40.818745 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:40.818811 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:40.818881 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:33:40.818915 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:33:40.819087 885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:40.819157 885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf3ddf8acd1465cb7317b00e8083649, State: Initialized, Role: FOLLOWER
I20260807 08:33:40.819553 885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } }
I20260807 08:33:40.819679 885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:40.819746 885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:40.819847 885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:40.820650 885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } }
I20260807 08:33:40.820955 885 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf3ddf8acd1465cb7317b00e8083649; no voters:
I20260807 08:33:40.821245 885 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:40.821321 898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:40.821539 898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [term 1 LEADER]: Becoming Leader. State: Replica: 7cf3ddf8acd1465cb7317b00e8083649, State: Running, Role: LEADER
I20260807 08:33:40.821851 898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } }
I20260807 08:33:40.822357 885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:40.823405 900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cf3ddf8acd1465cb7317b00e8083649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } } }
I20260807 08:33:40.823585 900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:40.823663 901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf3ddf8acd1465cb7317b00e8083649. Latest consensus state: current_term: 1 leader_uuid: "7cf3ddf8acd1465cb7317b00e8083649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf3ddf8acd1465cb7317b00e8083649" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38549 } } }
I20260807 08:33:40.823755 901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:40.824620 916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:40.827149 916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:40.832137 916 catalog_manager.cc:1347] Generated new cluster ID: 13cb14c2dc804cf19e026d25ed3ee6a9
I20260807 08:33:40.832197 916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:40.851186 916 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:40.851475 916 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:40.855683 916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cf3ddf8acd1465cb7317b00e8083649: Generated new TSK 0
I20260807 08:33:40.856257 916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:40.864689 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 794
2026-08-07T08:33:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:40 dist-test-slave-6zbq krb5kdc[929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:40 dist-test-slave-6zbq krb5kdc[929](info): set up 2 sockets
Aug 07 08:33:40 dist-test-slave-6zbq krb5kdc[929](info): commencing operation
krb5kdc: starting...
W20260807 08:33:41.977811 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:42.088943 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:42.193686 949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:42.193919 949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:42.193976 949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:42.197863 949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:42.197921 949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:42.197948 949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:42.197971 949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:42.197999 949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:42.203027 949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:42.203962 949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:42.204895 949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:42.212656 959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:42.212667 955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:42.212757 949 server_base.cc:1034] running on GCE node
W20260807 08:33:42.212661 956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:42.213447 949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:42.214406 949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:42.215564 949 hybrid_clock.cc:648] HybridClock initialized: now 1786091622215548 us; error 31 us; skew 500 ppm
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091622, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:42.218163 949 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:42.218991 949 webserver.cc:460] Webserver started at http://127.0.0.1:42731/ using document root <none> and password file <none>
I20260807 08:33:42.219465 949 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:42.219520 949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:42.219676 949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:42.221514 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "6342ab275af24af7b88c84cdd0596bd2"
format_stamp: "Formatted at 2026-08-07 08:33:42 on dist-test-slave-6zbq"
server_key: "87da0626cb5e8cb3ce787bdf0d18c96d"
server_key_iv: "c0c7b97e73c963c844f864171bc10d20"
server_key_version: "encryptionkey@0"
I20260807 08:33:42.221997 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "6342ab275af24af7b88c84cdd0596bd2"
format_stamp: "Formatted at 2026-08-07 08:33:42 on dist-test-slave-6zbq"
server_key: "87da0626cb5e8cb3ce787bdf0d18c96d"
server_key_iv: "c0c7b97e73c963c844f864171bc10d20"
server_key_version: "encryptionkey@0"
I20260807 08:33:42.225299 949 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:33:42.227476 969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:42.228538 949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:33:42.228638 949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "6342ab275af24af7b88c84cdd0596bd2"
format_stamp: "Formatted at 2026-08-07 08:33:42 on dist-test-slave-6zbq"
server_key: "87da0626cb5e8cb3ce787bdf0d18c96d"
server_key_iv: "c0c7b97e73c963c844f864171bc10d20"
server_key_version: "encryptionkey@0"
I20260807 08:33:42.228729 949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:42.245412 949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:42.248898 949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:42.249066 949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:42.256552 949 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37673
I20260807 08:33:42.256567 1047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37673 every 8 connection(s)
I20260807 08:33:42.257472 949 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:42.260149 1048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:42.264173 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 949
I20260807 08:33:42.264321 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:42.264601 1653 external_mini_cluster.cc:1428] Setting key adf02c0ce174a699e45251f52732e347
I20260807 08:33:42.265810 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: Bootstrap starting.
I20260807 08:33:42.268069 1048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:42.268854 1048 log.cc:824] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:42.270723 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: No bootstrap required, opened a new log
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:33:42.275342 1048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260807 08:33:42.275663 1048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:42.275723 1048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6342ab275af24af7b88c84cdd0596bd2, State: Initialized, Role: FOLLOWER
I20260807 08:33:42.276345 1048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260807 08:33:42.276543 1048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:42.276619 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:42.276728 1048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:42.278096 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260807 08:33:42.278537 1048 leader_election.cc:302] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6342ab275af24af7b88c84cdd0596bd2; no voters:
I20260807 08:33:42.278900 1048 leader_election.cc:288] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:42.279011 1053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:42.279258 1053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 6342ab275af24af7b88c84cdd0596bd2, State: Running, Role: LEADER
I20260807 08:33:42.279626 1053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260807 08:33:42.279714 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:42.265885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:53473 (local address 127.0.0.1:37673)
0807 08:33:42.266284 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:42.266299 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:42.266333 (+ 34us) server_negotiation.cc:407] Connection header received
0807 08:33:42.266962 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:42.266978 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:42.267307 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:42.267564 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:42.268416 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:42.269406 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:42.270301 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:42.270809 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:42.270953 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:42.273815 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:42.273822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:42.273842 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:42.273881 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:42.275858 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:42.276450 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:42.276456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:42.276463 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:42.276620 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:42.277128 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:42.277132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:42.277134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:42.277684 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:42.278061 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:42.278302 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 08:33:42.278595 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
I20260807 08:33:42.280237 1048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:42.282233 1054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6342ab275af24af7b88c84cdd0596bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } } }
I20260807 08:33:42.282407 1054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:42.282510 1055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6342ab275af24af7b88c84cdd0596bd2. Latest consensus state: current_term: 1 leader_uuid: "6342ab275af24af7b88c84cdd0596bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6342ab275af24af7b88c84cdd0596bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } } }
I20260807 08:33:42.282613 1055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:42.283394 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 08:33:42.285210 1070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:42.285339 1070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:42.285467 1071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:42.286439 1071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:42.292472 1071 catalog_manager.cc:1347] Generated new cluster ID: 2040989928eb4ce59d260027e0266c00
I20260807 08:33:42.292605 1071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:42.294076 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:42.285307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:41281 (local address 127.0.0.1:37673)
0807 08:33:42.285408 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:42.285412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:42.285631 (+ 219us) server_negotiation.cc:407] Connection header received
0807 08:33:42.285771 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:42.285774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:42.285821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:42.285882 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:42.286630 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:42.287178 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:42.287998 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:42.288263 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:42.288349 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:42.289989 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:42.289993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:42.289996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:42.290023 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:42.292122 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:42.292739 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:42.292742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:42.292744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:42.292832 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:42.293217 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:42.293221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:42.293222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:42.293437 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:42.293568 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:42.293844 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 08:33:42.293935 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 08:33:42.314085 1071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:42.315116 1071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:42.322916 1071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6342ab275af24af7b88c84cdd0596bd2: Generated new TSK 0
I20260807 08:33:42.323617 1071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:42.331983 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 949
W20260807 08:33:42.345515 926 connection.cc:535] client connection to 127.0.0.1:37673 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37673): Connection reset by peer (error 104)
2026-08-07T08:33:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[1088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[1088](info): set up 2 sockets
Aug 07 08:33:42 dist-test-slave-6zbq krb5kdc[1088](info): commencing operation
krb5kdc: starting...
W20260807 08:33:43.476056 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:43 dist-test-slave-6zbq krb5kdc[1088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:43.587225 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:43.691396 1103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:43.691635 1103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:43.691680 1103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:43.695483 1103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:43.695529 1103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:43.695549 1103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:43.695578 1103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:43.695604 1103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:43.700536 1103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:43.701366 1103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:43.702234 1103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:43.709693 1116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:43.709721 1112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:43.709719 1113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:43.709758 1103 server_base.cc:1034] running on GCE node
I20260807 08:33:43.710436 1103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:43.711338 1103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:43.712484 1103 hybrid_clock.cc:648] HybridClock initialized: now 1786091623712465 us; error 33 us; skew 500 ppm
Aug 07 08:33:43 dist-test-slave-6zbq krb5kdc[1088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:43.714920 1103 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:43.715677 1103 webserver.cc:460] Webserver started at http://127.0.0.1:33737/ using document root <none> and password file <none>
I20260807 08:33:43.716121 1103 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:43.716174 1103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:43.716321 1103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:43.718053 1103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80"
format_stamp: "Formatted at 2026-08-07 08:33:43 on dist-test-slave-6zbq"
server_key: "7621c8c578558e2af89782d9b0b83d73"
server_key_iv: "189e5e34a937aa33fcf83a701274ebc9"
server_key_version: "encryptionkey@0"
I20260807 08:33:43.718547 1103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80"
format_stamp: "Formatted at 2026-08-07 08:33:43 on dist-test-slave-6zbq"
server_key: "7621c8c578558e2af89782d9b0b83d73"
server_key_iv: "189e5e34a937aa33fcf83a701274ebc9"
server_key_version: "encryptionkey@0"
I20260807 08:33:43.721707 1103 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:33:43.723817 1129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:43.724858 1103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:33:43.724961 1103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80"
format_stamp: "Formatted at 2026-08-07 08:33:43 on dist-test-slave-6zbq"
server_key: "7621c8c578558e2af89782d9b0b83d73"
server_key_iv: "189e5e34a937aa33fcf83a701274ebc9"
server_key_version: "encryptionkey@0"
I20260807 08:33:43.725046 1103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:43.747846 1103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:43.750123 1103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:43.750277 1103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:43.756505 1103 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36799
I20260807 08:33:43.756536 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36799 every 8 connection(s)
I20260807 08:33:43.757313 1103 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:43.759884 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:43.762436 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1103
I20260807 08:33:43.762571 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:43.762871 1653 external_mini_cluster.cc:1428] Setting key 5c0be2ef527fa400d2bda8f39a921759
I20260807 08:33:43.765379 1205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80: Bootstrap starting.
I20260807 08:33:43.767416 1205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:43.768074 1205 log.cc:824] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:43.769820 1205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80: No bootstrap required, opened a new log
I20260807 08:33:43.770752 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:43.764039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:56923 (local address 127.0.0.1:36799)
0807 08:33:43.764394 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:43.764409 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:43.764440 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:33:43.765020 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:43.765035 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:43.765335 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:43.765630 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:43.765998 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:43.766994 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:43.767870 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:43.768337 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:43.768474 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:43.769258 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:43.769263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:43.769274 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:43.769603 (+ 329us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:43.769653 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:43.769684 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:43.769793 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:43.769978 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 08:33:43.770092 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260807 08:33:43.773339 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:43.773581 1205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260807 08:33:43.773782 1205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:43.773835 1205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7c67f6c4f5468b828dd1c5e1e7ac80, State: Initialized, Role: FOLLOWER
I20260807 08:33:43.774186 1205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260807 08:33:43.774315 1205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:43.774427 1205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:43.774533 1205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:43.775400 1205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260807 08:33:43.775719 1205 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7c67f6c4f5468b828dd1c5e1e7ac80; no voters:
I20260807 08:33:43.776006 1205 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:43.776073 1217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:43.776274 1217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7c67f6c4f5468b828dd1c5e1e7ac80, State: Running, Role: LEADER
I20260807 08:33:43.776602 1217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260807 08:33:43.777168 1205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:43.778496 1220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } } }
I20260807 08:33:43.778620 1220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:43.778976 1221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7c67f6c4f5468b828dd1c5e1e7ac80. Latest consensus state: current_term: 1 leader_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7c67f6c4f5468b828dd1c5e1e7ac80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } } }
I20260807 08:33:43.779110 1221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:43.779258 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:43.775157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:55563 (local address 127.0.0.1:36799)
0807 08:33:43.775252 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:43.775255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:43.775364 (+ 109us) server_negotiation.cc:407] Connection header received
0807 08:33:43.775476 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:43.775477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:43.775516 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:43.775587 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:43.775832 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:43.776408 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:43.777226 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:43.777551 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:43.777639 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:43.778578 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:43.778581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:43.778584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:43.778596 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:43.778620 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:43.778632 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:43.778706 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:43.779101 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 08:33:43.779133 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:33:43.779590 1241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:43.782742 1241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:43.787518 1241 catalog_manager.cc:1347] Generated new cluster ID: 74ff0b8ad53543b68055f2181b30e4f6
I20260807 08:33:43.787624 1241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:43.793656 1241 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:43.794533 1241 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:43.800586 1241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7c67f6c4f5468b828dd1c5e1e7ac80: Generated new TSK 0
I20260807 08:33:43.801234 1241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:43.812774 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1103
W20260807 08:33:43.824214 1084 connection.cc:535] client connection to 127.0.0.1:36799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36799): Connection reset by peer (error 104)
2026-08-07T08:33:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:43 dist-test-slave-6zbq krb5kdc[1256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:43 dist-test-slave-6zbq krb5kdc[1256](info): set up 2 sockets
Aug 07 08:33:43 dist-test-slave-6zbq krb5kdc[1256](info): commencing operation
krb5kdc: starting...
W20260807 08:33:44.938584 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:45 dist-test-slave-6zbq krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:45.051066 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:45.155063 1276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:45.155289 1276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:45.155328 1276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:45.159103 1276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:45.159145 1276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:45.159168 1276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:45.159183 1276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:45.159201 1276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:45.164028 1276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:45.164819 1276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:45.165647 1276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:45.173082 1285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:45.173084 1282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:45.173076 1283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:45.173252 1276 server_base.cc:1034] running on GCE node
I20260807 08:33:45.173818 1276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:45.174804 1276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:45.175957 1276 hybrid_clock.cc:648] HybridClock initialized: now 1786091625175943 us; error 25 us; skew 500 ppm
Aug 07 08:33:45 dist-test-slave-6zbq krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:45.181710 1276 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:45.182498 1276 webserver.cc:460] Webserver started at http://127.0.0.1:43793/ using document root <none> and password file <none>
I20260807 08:33:45.182962 1276 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:45.183015 1276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:45.183171 1276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:45.184947 1276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "d58fdc016d73467bb555f112dd93e229"
format_stamp: "Formatted at 2026-08-07 08:33:45 on dist-test-slave-6zbq"
server_key: "75382ebf182e2ab9eb542956f850fde4"
server_key_iv: "fe5e21ee8ae91e8415a40b34235a3a0f"
server_key_version: "encryptionkey@0"
I20260807 08:33:45.185427 1276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "d58fdc016d73467bb555f112dd93e229"
format_stamp: "Formatted at 2026-08-07 08:33:45 on dist-test-slave-6zbq"
server_key: "75382ebf182e2ab9eb542956f850fde4"
server_key_iv: "fe5e21ee8ae91e8415a40b34235a3a0f"
server_key_version: "encryptionkey@0"
I20260807 08:33:45.188760 1276 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 08:33:45.190900 1294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:45.191954 1276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:45.192070 1276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "d58fdc016d73467bb555f112dd93e229"
format_stamp: "Formatted at 2026-08-07 08:33:45 on dist-test-slave-6zbq"
server_key: "75382ebf182e2ab9eb542956f850fde4"
server_key_iv: "fe5e21ee8ae91e8415a40b34235a3a0f"
server_key_version: "encryptionkey@0"
I20260807 08:33:45.192157 1276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:45.196324 1276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:45.198737 1276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:45.198884 1276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:45.205135 1276 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43211
I20260807 08:33:45.205186 1374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43211 every 8 connection(s)
I20260807 08:33:45.205961 1276 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:45.208431 1375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:45.213416 1375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229: Bootstrap starting.
I20260807 08:33:45.215651 1375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:45.215977 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1276
I20260807 08:33:45.216102 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:45.216343 1653 external_mini_cluster.cc:1428] Setting key 5f12049532040093c17e037cd27ad7ce
I20260807 08:33:45.216346 1375 log.cc:824] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:45.218075 1375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229: No bootstrap required, opened a new log
I20260807 08:33:45.220867 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:45.217226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40159 (local address 127.0.0.1:43211)
0807 08:33:45.217592 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:45.217611 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:45.217794 (+ 183us) server_negotiation.cc:407] Connection header received
0807 08:33:45.218532 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:45.218550 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:45.218806 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:45.219085 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:45.219789 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:45.219793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:45.219804 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:45.219815 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:45.219866 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:45.219884 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:45.219973 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:45.220081 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 08:33:45.220228 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260807 08:33:45.221581 1375 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } }
I20260807 08:33:45.221776 1375 raft_consensus.cc:374] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:45.221840 1375 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58fdc016d73467bb555f112dd93e229, State: Initialized, Role: FOLLOWER
I20260807 08:33:45.222213 1375 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } }
I20260807 08:33:45.222353 1375 raft_consensus.cc:388] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:45.222441 1375 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:45.222579 1375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:45.223338 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:45.223410 1375 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } }
I20260807 08:33:45.223721 1375 leader_election.cc:302] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d58fdc016d73467bb555f112dd93e229; no voters:
I20260807 08:33:45.223966 1375 leader_election.cc:288] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:45.224071 1385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:45.224274 1385 raft_consensus.cc:686] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [term 1 LEADER]: Becoming Leader. State: Replica: d58fdc016d73467bb555f112dd93e229, State: Running, Role: LEADER
I20260807 08:33:45.224632 1385 consensus_queue.cc:237] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } }
I20260807 08:33:45.225032 1375 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:45.225932 1386 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58fdc016d73467bb555f112dd93e229. Latest consensus state: current_term: 1 leader_uuid: "d58fdc016d73467bb555f112dd93e229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } } }
I20260807 08:33:45.226058 1386 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:45.226495 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:45.226724 1388 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d58fdc016d73467bb555f112dd93e229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d58fdc016d73467bb555f112dd93e229" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43211 } } }
I20260807 08:33:45.226809 1388 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:45.227334 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:45.225384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:56911 (local address 127.0.0.1:43211)
0807 08:33:45.225472 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:45.225476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:45.225792 (+ 316us) server_negotiation.cc:407] Connection header received
0807 08:33:45.225978 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:45.225981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:45.226015 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:45.226093 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:45.226882 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:45.226885 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:45.226887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:45.226899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:45.226924 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:45.226937 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:45.226993 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:45.227190 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 08:33:45.227239 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:33:45.229525 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:45.234416 1411 catalog_manager.cc:1347] Generated new cluster ID: 2e21eb2628a344e699ad3d81158588c9
I20260807 08:33:45.234468 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:45.276681 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:45.276968 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:45.282851 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d58fdc016d73467bb555f112dd93e229: Generated new TSK 0
I20260807 08:33:45.283427 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:45.292513 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1276
2026-08-07T08:33:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:45 dist-test-slave-6zbq krb5kdc[1432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:45 dist-test-slave-6zbq krb5kdc[1432](info): set up 2 sockets
Aug 07 08:33:45 dist-test-slave-6zbq krb5kdc[1432](info): commencing operation
krb5kdc: starting...
W20260807 08:33:46.410192 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:46.521028 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:46.628839 1454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:46.629061 1454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:46.629106 1454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:46.632992 1454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:46.633044 1454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:46.633064 1454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:46.633092 1454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:46.633121 1454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:46.638178 1454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:46.639088 1454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:46.639990 1454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:46.647821 1460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:46.647831 1459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:46.647966 1454 server_base.cc:1034] running on GCE node
W20260807 08:33:46.647868 1463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:46.648612 1454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:46.649586 1454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:46.650743 1454 hybrid_clock.cc:648] HybridClock initialized: now 1786091626650729 us; error 26 us; skew 500 ppm
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091626, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:46.653291 1454 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:46.654081 1454 webserver.cc:460] Webserver started at http://127.0.0.1:33557/ using document root <none> and password file <none>
I20260807 08:33:46.654610 1454 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:46.654675 1454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:46.654841 1454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:46.656677 1454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "a4c9a300771045699e8fd7db624cb75d"
format_stamp: "Formatted at 2026-08-07 08:33:46 on dist-test-slave-6zbq"
server_key: "0b87cb1413ed6b4167db09e9223d831d"
server_key_iv: "01d59f0250fa65a5034882b0eb2d2766"
server_key_version: "encryptionkey@0"
I20260807 08:33:46.657173 1454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "a4c9a300771045699e8fd7db624cb75d"
format_stamp: "Formatted at 2026-08-07 08:33:46 on dist-test-slave-6zbq"
server_key: "0b87cb1413ed6b4167db09e9223d831d"
server_key_iv: "01d59f0250fa65a5034882b0eb2d2766"
server_key_version: "encryptionkey@0"
I20260807 08:33:46.660785 1454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:33:46.663012 1473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:46.664094 1454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:46.664217 1454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "a4c9a300771045699e8fd7db624cb75d"
format_stamp: "Formatted at 2026-08-07 08:33:46 on dist-test-slave-6zbq"
server_key: "0b87cb1413ed6b4167db09e9223d831d"
server_key_iv: "01d59f0250fa65a5034882b0eb2d2766"
server_key_version: "encryptionkey@0"
I20260807 08:33:46.664309 1454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:46.682633 1454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:46.684967 1454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:46.685134 1454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:46.691406 1454 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33075
I20260807 08:33:46.691437 1554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33075 every 8 connection(s)
I20260807 08:33:46.692219 1454 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:46.694648 1556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:46.696331 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1454
I20260807 08:33:46.696446 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:46.696697 1653 external_mini_cluster.cc:1428] Setting key 21ade13e39c7416b4df123c30817a937
I20260807 08:33:46.700812 1556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d: Bootstrap starting.
I20260807 08:33:46.703233 1556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:46.704025 1556 log.cc:824] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d: Log is configured to *not* fsync() on all Append() calls
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 08:33:46.705709 1556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d: No bootstrap required, opened a new log
I20260807 08:33:46.709930 1556 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 08:33:46.710187 1556 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:46.710266 1556 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c9a300771045699e8fd7db624cb75d, State: Initialized, Role: FOLLOWER
I20260807 08:33:46.710541 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:46.697922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:38145 (local address 127.0.0.1:33075)
0807 08:33:46.698315 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:46.698334 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:46.698400 (+ 66us) server_negotiation.cc:407] Connection header received
0807 08:33:46.699077 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:46.699101 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:46.699500 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:46.699853 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:46.700468 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:46.701861 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:46.702740 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:46.703393 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:46.703561 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:46.705818 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:46.705824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:46.705839 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:46.705867 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:46.707627 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:46.708119 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:46.708125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:46.708131 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:46.708267 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:46.708528 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:46.708532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:46.708533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:46.709000 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:46.709213 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:46.709375 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 08:33:46.709586 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
I20260807 08:33:46.710723 1556 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 08:33:46.710875 1556 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:46.710940 1556 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:46.711033 1556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:46.711925 1556 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 08:33:46.712270 1556 leader_election.cc:302] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4c9a300771045699e8fd7db624cb75d; no voters:
I20260807 08:33:46.712570 1556 leader_election.cc:288] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:46.712674 1561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:46.712878 1561 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [term 1 LEADER]: Becoming Leader. State: Replica: a4c9a300771045699e8fd7db624cb75d, State: Running, Role: LEADER
I20260807 08:33:46.713207 1561 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 08:33:46.713352 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:46.713707 1556 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:46.714771 1562 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4c9a300771045699e8fd7db624cb75d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } } }
I20260807 08:33:46.714900 1562 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:46.715473 1565 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4c9a300771045699e8fd7db624cb75d. Latest consensus state: current_term: 1 leader_uuid: "a4c9a300771045699e8fd7db624cb75d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a300771045699e8fd7db624cb75d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } } }
I20260807 08:33:46.715653 1565 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:46.716199 1580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:46.719359 1580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:46.724431 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:46.715800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:42585 (local address 127.0.0.1:33075)
0807 08:33:46.715903 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:46.715907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:46.716612 (+ 705us) server_negotiation.cc:407] Connection header received
0807 08:33:46.716719 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:46.716722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:46.716763 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:46.716845 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:46.717395 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:46.718094 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:46.719209 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:46.719545 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:46.719640 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:46.721265 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:46.721269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:46.721271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:46.721295 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:46.722563 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:46.723018 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:46.723022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:46.723023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:46.723096 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:46.723397 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:46.723400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:46.723402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:46.723557 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:46.724097 (+ 540us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:46.724195 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 08:33:46.724286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:33:46.725660 1580 catalog_manager.cc:1347] Generated new cluster ID: 0f237e874c804f289f1dcf683365989d
I20260807 08:33:46.725732 1580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:46.742389 1580 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:46.743188 1580 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:46.764621 1580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4c9a300771045699e8fd7db624cb75d: Generated new TSK 0
I20260807 08:33:46.765336 1580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:46.772341 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1454
W20260807 08:33:46.786564 1428 connection.cc:535] client connection to 127.0.0.1:33075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33075): Connection reset by peer (error 104)
2026-08-07T08:33:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1601](info): set up 2 sockets
Aug 07 08:33:46 dist-test-slave-6zbq krb5kdc[1601](info): commencing operation
krb5kdc: starting...
W20260807 08:33:47.897532 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:47 dist-test-slave-6zbq krb5kdc[1601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:48.006129 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:48.110091 1622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:48.110319 1622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:48.110396 1622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:48.114661 1622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:48.114723 1622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:48.114751 1622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:48.114774 1622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:48.114804 1622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:48.120002 1622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:48.120903 1622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:48.121829 1622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:48.129946 1628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:48.130066 1622 server_base.cc:1034] running on GCE node
W20260807 08:33:48.129969 1627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:48.130263 1631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:48.130808 1622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:48.131819 1622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:48.132970 1622 hybrid_clock.cc:648] HybridClock initialized: now 1786091628132948 us; error 33 us; skew 500 ppm
Aug 07 08:33:48 dist-test-slave-6zbq krb5kdc[1601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:48.135458 1622 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:48.136233 1622 webserver.cc:460] Webserver started at http://127.0.0.1:38857/ using document root <none> and password file <none>
I20260807 08:33:48.136718 1622 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:48.136788 1622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:48.136981 1622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:48.138998 1622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "f5314c66868e4c098b39219fbb7e7491"
format_stamp: "Formatted at 2026-08-07 08:33:48 on dist-test-slave-6zbq"
server_key: "d55d6504ca7ae8cfcfadd84cd81de9de"
server_key_iv: "ecba085313c1aeeb0a6a88044478b59c"
server_key_version: "encryptionkey@0"
I20260807 08:33:48.139508 1622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "f5314c66868e4c098b39219fbb7e7491"
format_stamp: "Formatted at 2026-08-07 08:33:48 on dist-test-slave-6zbq"
server_key: "d55d6504ca7ae8cfcfadd84cd81de9de"
server_key_iv: "ecba085313c1aeeb0a6a88044478b59c"
server_key_version: "encryptionkey@0"
I20260807 08:33:48.142969 1622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:33:48.145141 1644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:48.146251 1622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:48.146377 1622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "f5314c66868e4c098b39219fbb7e7491"
format_stamp: "Formatted at 2026-08-07 08:33:48 on dist-test-slave-6zbq"
server_key: "d55d6504ca7ae8cfcfadd84cd81de9de"
server_key_iv: "ecba085313c1aeeb0a6a88044478b59c"
server_key_version: "encryptionkey@0"
I20260807 08:33:48.146467 1622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:48.164400 1622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:48.166832 1622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:48.166989 1622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:48.173368 1622 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33839
I20260807 08:33:48.173380 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33839 every 8 connection(s)
I20260807 08:33:48.174192 1622 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:48.176736 1728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:48.181133 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1622
I20260807 08:33:48.181274 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:48.181509 1653 external_mini_cluster.cc:1428] Setting key ff774f2ee050c2e5e587f266f237c3f4
I20260807 08:33:48.182026 1728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: Bootstrap starting.
I20260807 08:33:48.184326 1728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:48.185034 1728 log.cc:824] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:48.186833 1728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: No bootstrap required, opened a new log
I20260807 08:33:48.189724 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:48.182731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:42381 (local address 127.0.0.1:33839)
0807 08:33:48.183124 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:48.183138 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:48.183171 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:48.183789 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:48.183805 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:48.184117 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:48.184399 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:48.184784 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:48.185910 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:48.186824 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:48.187301 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:48.187443 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:48.188248 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:48.188254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:48.188265 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:48.188606 (+ 341us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:48.188657 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:48.188678 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:48.188791 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:48.188963 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 08:33:48.189084 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":113,"threads_started":1}
I20260807 08:33:48.190466 1728 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260807 08:33:48.190732 1728 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:48.190796 1728 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5314c66868e4c098b39219fbb7e7491, State: Initialized, Role: FOLLOWER
I20260807 08:33:48.191186 1728 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260807 08:33:48.191309 1728 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:48.191370 1728 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:48.191470 1728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:48.192373 1728 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260807 08:33:48.192529 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:48.192719 1728 leader_election.cc:302] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5314c66868e4c098b39219fbb7e7491; no voters:
I20260807 08:33:48.192994 1728 leader_election.cc:288] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:48.193096 1736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:48.193322 1736 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [term 1 LEADER]: Becoming Leader. State: Replica: f5314c66868e4c098b39219fbb7e7491, State: Running, Role: LEADER
I20260807 08:33:48.193650 1736 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } }
I20260807 08:33:48.194193 1728 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:48.195115 1738 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5314c66868e4c098b39219fbb7e7491. Latest consensus state: current_term: 1 leader_uuid: "f5314c66868e4c098b39219fbb7e7491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } } }
I20260807 08:33:48.195261 1738 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:48.195538 1737 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5314c66868e4c098b39219fbb7e7491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5314c66868e4c098b39219fbb7e7491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33839 } } }
I20260807 08:33:48.195623 1737 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:48.198876 1766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:48.198956 1766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:48.199033 1757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:48.199406 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:48.194468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:36367 (local address 127.0.0.1:33839)
0807 08:33:48.194571 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:48.194576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:48.194744 (+ 168us) server_negotiation.cc:407] Connection header received
0807 08:33:48.194843 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:48.194846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:48.194884 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:48.194945 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:48.195845 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:48.196540 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:48.197778 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:48.198062 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:48.198152 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:48.198922 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:48.198926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:48.198928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:48.198941 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:48.198966 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:48.198979 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:48.199055 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:48.199249 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:33:48.199281 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 08:33:48.200047 1757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:48.205204 1757 catalog_manager.cc:1347] Generated new cluster ID: 72663aea8994467896fce40da7d325e3
I20260807 08:33:48.205255 1757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:48.225138 1757 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:48.225993 1757 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:48.230526 1757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5314c66868e4c098b39219fbb7e7491: Generated new TSK 0
I20260807 08:33:48.231076 1757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:48.234477 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1622
W20260807 08:33:48.248466 1597 connection.cc:535] client connection to 127.0.0.1:33839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33839): Connection reset by peer (error 104)
2026-08-07T08:33:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:48 dist-test-slave-6zbq krb5kdc[1775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:48 dist-test-slave-6zbq krb5kdc[1775](info): set up 2 sockets
Aug 07 08:33:48 dist-test-slave-6zbq krb5kdc[1775](info): commencing operation
krb5kdc: starting...
W20260807 08:33:49.351105 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:49 dist-test-slave-6zbq krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 08:33:49.464540 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:49.570004 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:49.570251 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:49.570309 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:49.574211 1792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:49.574267 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:49.574294 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:49.574316 1792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:49.574358 1792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:49.579476 1792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:49.580417 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:49.581377 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:49.589324 1803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:49.589324 1806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:49.589326 1802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:49.589681 1792 server_base.cc:1034] running on GCE node
I20260807 08:33:49.590224 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:49.591272 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:49.592440 1792 hybrid_clock.cc:648] HybridClock initialized: now 1786091629592424 us; error 27 us; skew 500 ppm
Aug 07 08:33:49 dist-test-slave-6zbq krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:49.595085 1792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 08:33:49.595932 1792 webserver.cc:460] Webserver started at http://127.0.0.1:35961/ using document root <none> and password file <none>
I20260807 08:33:49.596479 1792 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:49.596545 1792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:49.596738 1792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:49.598714 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "57af9ba3d78a47afb0b514917b3db8b8"
format_stamp: "Formatted at 2026-08-07 08:33:49 on dist-test-slave-6zbq"
server_key: "333c46dc6398b5b76c28ac5daa4bec16"
server_key_iv: "8d17ca8dbd6c30ef43f2d064f651d0af"
server_key_version: "encryptionkey@0"
I20260807 08:33:49.599241 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "57af9ba3d78a47afb0b514917b3db8b8"
format_stamp: "Formatted at 2026-08-07 08:33:49 on dist-test-slave-6zbq"
server_key: "333c46dc6398b5b76c28ac5daa4bec16"
server_key_iv: "8d17ca8dbd6c30ef43f2d064f651d0af"
server_key_version: "encryptionkey@0"
I20260807 08:33:49.602713 1792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 08:33:49.605027 1819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:49.606129 1792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:49.606244 1792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "57af9ba3d78a47afb0b514917b3db8b8"
format_stamp: "Formatted at 2026-08-07 08:33:49 on dist-test-slave-6zbq"
server_key: "333c46dc6398b5b76c28ac5daa4bec16"
server_key_iv: "8d17ca8dbd6c30ef43f2d064f651d0af"
server_key_version: "encryptionkey@0"
I20260807 08:33:49.606333 1792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:49.610575 1792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:49.613054 1792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:49.613206 1792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:49.619534 1792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35127
I20260807 08:33:49.619552 1888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35127 every 8 connection(s)
I20260807 08:33:49.620347 1792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:49.622936 1889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:49.627980 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8: Bootstrap starting.
I20260807 08:33:49.629880 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1792
I20260807 08:33:49.630009 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:49.630244 1653 external_mini_cluster.cc:1428] Setting key 19166cf649b29f9d460286778061c63c
I20260807 08:33:49.630268 1889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:49.631024 1889 log.cc:824] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:49.632843 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8: No bootstrap required, opened a new log
I20260807 08:33:49.634680 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:49.631242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:48479 (local address 127.0.0.1:35127)
0807 08:33:49.631602 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:49.631621 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:49.631662 (+ 41us) server_negotiation.cc:407] Connection header received
0807 08:33:49.632310 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:49.632326 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:49.632571 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:49.632857 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:49.633564 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:49.633569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:49.633582 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:49.633592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:49.633642 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:49.633660 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:49.633752 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:49.633859 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 08:33:49.634005 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":109,"threads_started":1}
I20260807 08:33:49.636605 1889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } }
I20260807 08:33:49.636850 1889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:49.636909 1889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57af9ba3d78a47afb0b514917b3db8b8, State: Initialized, Role: FOLLOWER
I20260807 08:33:49.637228 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:49.637302 1889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } }
I20260807 08:33:49.637449 1889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:49.637511 1889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:49.637609 1889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:49.638397 1889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } }
I20260807 08:33:49.638715 1889 leader_election.cc:302] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57af9ba3d78a47afb0b514917b3db8b8; no voters:
I20260807 08:33:49.639008 1889 leader_election.cc:288] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:49.639096 1904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:49.639441 1904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [term 1 LEADER]: Becoming Leader. State: Replica: 57af9ba3d78a47afb0b514917b3db8b8, State: Running, Role: LEADER
I20260807 08:33:49.639739 1904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } }
I20260807 08:33:49.640260 1889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:49.640842 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:49.639131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:42803 (local address 127.0.0.1:35127)
0807 08:33:49.639216 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:49.639220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:49.639457 (+ 237us) server_negotiation.cc:407] Connection header received
0807 08:33:49.639602 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:49.639605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:49.639638 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:49.639718 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:49.640417 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:49.640421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:49.640423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:49.640435 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:49.640461 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:49.640473 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:49.640558 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:49.640701 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 08:33:49.640736 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 08:33:49.640923 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57af9ba3d78a47afb0b514917b3db8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } } }
I20260807 08:33:49.641067 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:49.641338 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57af9ba3d78a47afb0b514917b3db8b8. Latest consensus state: current_term: 1 leader_uuid: "57af9ba3d78a47afb0b514917b3db8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57af9ba3d78a47afb0b514917b3db8b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35127 } } }
I20260807 08:33:49.641474 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:49.641824 1917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:49.644465 1917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:49.649564 1917 catalog_manager.cc:1347] Generated new cluster ID: b1c4ef0cee11494b95758c44a5b82b2a
I20260807 08:33:49.649624 1917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:49.670719 1917 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:49.670967 1917 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:49.678381 1917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57af9ba3d78a47afb0b514917b3db8b8: Generated new TSK 0
I20260807 08:33:49.678916 1917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:49.684255 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1792
2026-08-07T08:33:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:49 dist-test-slave-6zbq krb5kdc[1935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:49 dist-test-slave-6zbq krb5kdc[1935](info): set up 2 sockets
Aug 07 08:33:49 dist-test-slave-6zbq krb5kdc[1935](info): commencing operation
krb5kdc: starting...
W20260807 08:33:50.806403 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:50 dist-test-slave-6zbq krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:50.919095 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46415
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:46415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:51.023936 1958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:51.024149 1958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:51.024191 1958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:51.028041 1958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:51.028087 1958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:51.028107 1958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:51.028137 1958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:51.028162 1958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:51.033085 1958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:46415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:46415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:51.033929 1958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:51.034818 1958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:51.042523 1974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:51.042538 1970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:51.042482 1968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:51.042732 1958 server_base.cc:1034] running on GCE node
I20260807 08:33:51.043251 1958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:51.044234 1958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:51.045382 1958 hybrid_clock.cc:648] HybridClock initialized: now 1786091631045365 us; error 37 us; skew 500 ppm
Aug 07 08:33:51 dist-test-slave-6zbq krb5kdc[1935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091631, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:51.052760 1958 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:51.053614 1958 webserver.cc:460] Webserver started at http://127.1.157.126:42147/ using document root <none> and password file <none>
I20260807 08:33:51.054131 1958 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:51.054198 1958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:51.054406 1958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:51.056309 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "cf42b2545f9b485e932c1eec4bc8246b"
format_stamp: "Formatted at 2026-08-07 08:33:51 on dist-test-slave-6zbq"
server_key: "b93ebc9d4cdcd58cc5ff3e8d23e18873"
server_key_iv: "109c50e445dbfbb90102a98d42d006f1"
server_key_version: "encryptionkey@0"
I20260807 08:33:51.056838 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "cf42b2545f9b485e932c1eec4bc8246b"
format_stamp: "Formatted at 2026-08-07 08:33:51 on dist-test-slave-6zbq"
server_key: "b93ebc9d4cdcd58cc5ff3e8d23e18873"
server_key_iv: "109c50e445dbfbb90102a98d42d006f1"
server_key_version: "encryptionkey@0"
I20260807 08:33:51.060249 1958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:33:51.062413 1982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:51.063473 1958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:51.063596 1958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "cf42b2545f9b485e932c1eec4bc8246b"
format_stamp: "Formatted at 2026-08-07 08:33:51 on dist-test-slave-6zbq"
server_key: "b93ebc9d4cdcd58cc5ff3e8d23e18873"
server_key_iv: "109c50e445dbfbb90102a98d42d006f1"
server_key_version: "encryptionkey@0"
I20260807 08:33:51.063683 1958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:51.082751 1958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:51.085522 1958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:51.085680 1958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:51.092242 1958 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:46415
I20260807 08:33:51.092258 2057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:46415 every 8 connection(s)
I20260807 08:33:51.093071 1958 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:51.094257 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 1958
I20260807 08:33:51.094434 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:51.094726 1653 external_mini_cluster.cc:1428] Setting key 931496b766f6ffa6efd514a709cba259
I20260807 08:33:51.095885 2059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:51.100955 2059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b: Bootstrap starting.
Aug 07 08:33:51 dist-test-slave-6zbq krb5kdc[1935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:51.103230 2059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:51.103973 2059 log.cc:824] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:51.105720 2059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b: No bootstrap required, opened a new log
I20260807 08:33:51.107707 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:51.095981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43919 (local address 127.1.157.126:46415)
0807 08:33:51.096383 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:51.096398 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:51.096431 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:51.097038 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:51.097053 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:51.097359 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:51.097639 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:51.098421 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:51.099371 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:51.100217 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:51.100672 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:51.100814 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:51.103329 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:51.103336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:51.103352 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:51.103379 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:51.105029 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:51.105522 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:51.105527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:51.105531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:51.105645 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:51.105984 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:51.105986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:51.105987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:51.106364 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:51.106564 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:51.106885 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 08:33:51.107080 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
I20260807 08:33:51.109269 2059 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } }
I20260807 08:33:51.109594 2059 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:51.109663 2059 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf42b2545f9b485e932c1eec4bc8246b, State: Initialized, Role: FOLLOWER
I20260807 08:33:51.110072 2059 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } }
I20260807 08:33:51.110395 2059 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:51.110479 2059 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:51.110579 2059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:51.110710 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:51.111411 2059 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } }
I20260807 08:33:51.111735 2059 leader_election.cc:302] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf42b2545f9b485e932c1eec4bc8246b; no voters:
I20260807 08:33:51.112066 2059 leader_election.cc:288] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:51.112179 2073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:51.112840 2073 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [term 1 LEADER]: Becoming Leader. State: Replica: cf42b2545f9b485e932c1eec4bc8246b, State: Running, Role: LEADER
I20260807 08:33:51.113118 2073 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } }
I20260807 08:33:51.113273 2059 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:51.114269 2073 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf42b2545f9b485e932c1eec4bc8246b. Latest consensus state: current_term: 1 leader_uuid: "cf42b2545f9b485e932c1eec4bc8246b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } } }
I20260807 08:33:51.114403 2073 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:51.114750 2086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:51.116530 2074 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf42b2545f9b485e932c1eec4bc8246b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42b2545f9b485e932c1eec4bc8246b" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 46415 } } }
I20260807 08:33:51.116648 2074 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:51.117653 2086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:51.121349 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:51.112550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47203 (local address 127.1.157.126:46415)
0807 08:33:51.112641 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:51.112645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:51.112819 (+ 174us) server_negotiation.cc:407] Connection header received
0807 08:33:51.112917 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:51.112920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:51.112957 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:51.113022 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:51.113749 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:51.114452 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:51.115536 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:51.115868 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:51.115969 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:51.117447 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:51.117451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:51.117453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:51.117477 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:51.119021 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:51.119586 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:51.119590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:51.119591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:51.119703 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:51.120104 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:51.120107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:51.120109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:51.120315 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:51.120453 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:51.121096 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 08:33:51.121194 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:33:51.122766 2086 catalog_manager.cc:1347] Generated new cluster ID: 6a323564a8224f748c0ff00670983146
I20260807 08:33:51.122828 2086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:51.152719 2086 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:51.153689 2086 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:51.159617 2086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf42b2545f9b485e932c1eec4bc8246b: Generated new TSK 0
I20260807 08:33:51.160180 2086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:51.167775 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 1958
W20260807 08:33:51.180732 1932 connection.cc:535] client connection to 127.1.157.126:46415 recv error: Network error: failed to read from TLS socket (remote: 127.1.157.126:46415): Connection reset by peer (error 104)
2026-08-07T08:33:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:51 dist-test-slave-6zbq krb5kdc[2101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:51 dist-test-slave-6zbq krb5kdc[2101](info): set up 2 sockets
Aug 07 08:33:51 dist-test-slave-6zbq krb5kdc[2101](info): commencing operation
krb5kdc: starting...
W20260807 08:33:52.297096 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:52 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:52.410794 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42397
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:52.516665 2121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:52.516906 2121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:52.516959 2121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:52.521193 2121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:52.521255 2121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:52.521286 2121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:52.521307 2121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:52.521327 2121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:52.526432 2121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42397
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:52.527361 2121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:52.528295 2121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:52.536062 2134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:52.536113 2121 server_base.cc:1034] running on GCE node
W20260807 08:33:52.536051 2130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:52.536065 2129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:52.536777 2121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:52.537750 2121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:52.538902 2121 hybrid_clock.cc:648] HybridClock initialized: now 1786091632538888 us; error 25 us; skew 500 ppm
Aug 07 08:33:52 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091632, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:52.541419 2121 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:52.542218 2121 webserver.cc:460] Webserver started at http://127.1.157.126:37605/ using document root <none> and password file <none>
I20260807 08:33:52.542743 2121 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:52.542809 2121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:52.542999 2121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:52.544855 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "85492af2bcb245708b8c7da301e27754"
format_stamp: "Formatted at 2026-08-07 08:33:52 on dist-test-slave-6zbq"
server_key: "4b49c116832e46d4a396c75f80a89667"
server_key_iv: "a1a38e5d83c1723c7fca3a619d3550de"
server_key_version: "encryptionkey@0"
I20260807 08:33:52.545454 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "85492af2bcb245708b8c7da301e27754"
format_stamp: "Formatted at 2026-08-07 08:33:52 on dist-test-slave-6zbq"
server_key: "4b49c116832e46d4a396c75f80a89667"
server_key_iv: "a1a38e5d83c1723c7fca3a619d3550de"
server_key_version: "encryptionkey@0"
I20260807 08:33:52.548817 2121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:33:52.550938 2143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:52.552038 2121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:52.552176 2121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "85492af2bcb245708b8c7da301e27754"
format_stamp: "Formatted at 2026-08-07 08:33:52 on dist-test-slave-6zbq"
server_key: "4b49c116832e46d4a396c75f80a89667"
server_key_iv: "a1a38e5d83c1723c7fca3a619d3550de"
server_key_version: "encryptionkey@0"
I20260807 08:33:52.552266 2121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:52.568681 2121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:52.571022 2121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:52.571184 2121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:52.577525 2121 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:42397
I20260807 08:33:52.577562 2227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:42397 every 8 connection(s)
I20260807 08:33:52.578567 2121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:52.581072 2228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:52.586012 2228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: Bootstrap starting.
I20260807 08:33:52.586035 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2121
I20260807 08:33:52.586170 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:52.586464 1653 external_mini_cluster.cc:1428] Setting key 6163eb3ca9046cfe89bced75aa82bc4d
I20260807 08:33:52.588394 2228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:52.589203 2228 log.cc:824] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:52.591270 2228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: No bootstrap required, opened a new log
I20260807 08:33:52.595228 2228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } }
I20260807 08:33:52.595479 2228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:52.595507 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:52.587639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49551 (local address 127.1.157.126:42397)
0807 08:33:52.588019 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:52.588037 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:52.588075 (+ 38us) server_negotiation.cc:407] Connection header received
0807 08:33:52.588848 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:52.588870 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:52.589257 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:52.589597 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:52.589907 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:52.591267 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:52.592129 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:52.592806 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:52.592963 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:52.593668 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:52.593675 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:52.593691 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:52.594171 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:52.594234 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:52.594262 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:52.594428 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:52.594545 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 08:33:52.594663 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":94,"threads_started":1}
I20260807 08:33:52.595541 2228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85492af2bcb245708b8c7da301e27754, State: Initialized, Role: FOLLOWER
I20260807 08:33:52.595920 2228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } }
I20260807 08:33:52.596046 2228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:52.596108 2228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:52.596200 2228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:52.596936 2228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } }
I20260807 08:33:52.597256 2228 leader_election.cc:302] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85492af2bcb245708b8c7da301e27754; no voters:
I20260807 08:33:52.597558 2228 leader_election.cc:288] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:52.597626 2234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:52.597823 2234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [term 1 LEADER]: Becoming Leader. State: Replica: 85492af2bcb245708b8c7da301e27754, State: Running, Role: LEADER
I20260807 08:33:52.598037 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:52.598735 2228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:52.600051 2234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } }
I20260807 08:33:52.601893 2239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85492af2bcb245708b8c7da301e27754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } } }
I20260807 08:33:52.601936 2237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85492af2bcb245708b8c7da301e27754. Latest consensus state: current_term: 1 leader_uuid: "85492af2bcb245708b8c7da301e27754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85492af2bcb245708b8c7da301e27754" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42397 } } }
I20260807 08:33:52.602011 2239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:52.602030 2237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:52.605273 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:52.600132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58091 (local address 127.1.157.126:42397)
0807 08:33:52.600242 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:52.600246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:52.600813 (+ 567us) server_negotiation.cc:407] Connection header received
0807 08:33:52.600919 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:52.600921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:52.600954 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:52.601012 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:52.601343 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:52.602076 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:52.603159 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:52.603424 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:52.603527 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:52.604556 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:52.604559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:52.604562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:52.604574 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:52.604626 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:52.604642 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:52.604731 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:52.605074 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 08:33:52.605136 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 08:33:52.605574 2263 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:52.605680 2263 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:52.607236 2262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:52.608144 2262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:52.613427 2262 catalog_manager.cc:1347] Generated new cluster ID: 6b7fe6e62934416db0468dd4d700dfb1
I20260807 08:33:52.613560 2262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:52.624912 2262 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:52.625669 2262 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:52.631541 2262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85492af2bcb245708b8c7da301e27754: Generated new TSK 0
I20260807 08:33:52.632045 2262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:52.642304 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2121
W20260807 08:33:52.656680 2097 connection.cc:535] client connection to 127.1.157.126:42397 recv error: Network error: failed to read from TLS socket (remote: 127.1.157.126:42397): Connection reset by peer (error 104)
2026-08-07T08:33:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:52 dist-test-slave-6zbq krb5kdc[2272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:52 dist-test-slave-6zbq krb5kdc[2272](info): set up 2 sockets
Aug 07 08:33:52 dist-test-slave-6zbq krb5kdc[2272](info): commencing operation
krb5kdc: starting...
W20260807 08:33:53.766044 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:53 dist-test-slave-6zbq krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:53.874874 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42585
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:38145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:53.979544 2291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:53.979764 2291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:53.979820 2291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:53.983645 2291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:53.983691 2291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:53.983718 2291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:53.983739 2291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:53.983767 2291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:53.988628 2291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:38145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:53.989480 2291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:53.990365 2291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:53.997865 2308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:53.997932 2291 server_base.cc:1034] running on GCE node
W20260807 08:33:53.997859 2303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:53.997876 2305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:53.998682 2291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:53.999646 2291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:54.000795 2291 hybrid_clock.cc:648] HybridClock initialized: now 1786091634000782 us; error 26 us; skew 500 ppm
Aug 07 08:33:54 dist-test-slave-6zbq krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091634, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:54.003348 2291 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:54.004103 2291 webserver.cc:460] Webserver started at http://127.1.157.126:45773/ using document root <none> and password file <none>
I20260807 08:33:54.004583 2291 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:54.004644 2291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:54.004815 2291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:54.006685 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "e74de1d5aa3e42f6a5839f42420d827c"
format_stamp: "Formatted at 2026-08-07 08:33:54 on dist-test-slave-6zbq"
server_key: "ea623b21e82b34e7ebf33dc93573ed88"
server_key_iv: "296d1566d20eda0e00c119a6f64d1560"
server_key_version: "encryptionkey@0"
I20260807 08:33:54.007187 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "e74de1d5aa3e42f6a5839f42420d827c"
format_stamp: "Formatted at 2026-08-07 08:33:54 on dist-test-slave-6zbq"
server_key: "ea623b21e82b34e7ebf33dc93573ed88"
server_key_iv: "296d1566d20eda0e00c119a6f64d1560"
server_key_version: "encryptionkey@0"
I20260807 08:33:54.010438 2291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:33:54.012498 2319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:54.013533 2291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:54.013638 2291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "e74de1d5aa3e42f6a5839f42420d827c"
format_stamp: "Formatted at 2026-08-07 08:33:54 on dist-test-slave-6zbq"
server_key: "ea623b21e82b34e7ebf33dc93573ed88"
server_key_iv: "296d1566d20eda0e00c119a6f64d1560"
server_key_version: "encryptionkey@0"
I20260807 08:33:54.013722 2291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:54.017863 2291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:54.020645 2291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:54.020808 2291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:54.027428 2291 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:42585
I20260807 08:33:54.027474 2399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:42585 every 8 connection(s)
I20260807 08:33:54.028249 2291 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:54.029531 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2291
I20260807 08:33:54.029665 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:54.029913 1653 external_mini_cluster.cc:1428] Setting key c048110bc2011ecdc1d917e31f59c7a2
I20260807 08:33:54.031019 2400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:54.034972 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:54.031164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59317 (local address 127.1.157.126:42585)
0807 08:33:54.031563 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:54.031580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:54.031619 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:33:54.032410 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:54.032432 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:54.032766 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:54.033094 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:54.033689 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:54.033695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:54.033708 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:54.033720 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:54.033775 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:54.033799 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:54.033932 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:54.033995 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 08:33:54.034130 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":130,"threads_started":1}
I20260807 08:33:54.037268 2400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c: Bootstrap starting.
I20260807 08:33:54.037547 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:54.039469 2400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:54.040175 2400 log.cc:824] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:54.040796 2404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:54.039357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57545 (local address 127.1.157.126:42585)
0807 08:33:54.039450 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:54.039454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:54.039645 (+ 191us) server_negotiation.cc:407] Connection header received
0807 08:33:54.039749 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:54.039752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:54.039780 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:54.039840 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:54.040467 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:54.040470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:54.040473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:54.040484 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:54.040509 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:54.040519 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:54.040594 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:54.040661 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 08:33:54.040695 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 08:33:54.042171 2400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c: No bootstrap required, opened a new log
I20260807 08:33:54.045478 2400 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } }
I20260807 08:33:54.045693 2400 raft_consensus.cc:374] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:54.045735 2400 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74de1d5aa3e42f6a5839f42420d827c, State: Initialized, Role: FOLLOWER
I20260807 08:33:54.046087 2400 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } }
I20260807 08:33:54.046211 2400 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:54.046252 2400 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:54.046319 2400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:54.047099 2400 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } }
I20260807 08:33:54.047372 2400 leader_election.cc:302] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74de1d5aa3e42f6a5839f42420d827c; no voters:
I20260807 08:33:54.047603 2400 leader_election.cc:288] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:54.047708 2415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:54.047907 2415 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [term 1 LEADER]: Becoming Leader. State: Replica: e74de1d5aa3e42f6a5839f42420d827c, State: Running, Role: LEADER
I20260807 08:33:54.048178 2415 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } }
I20260807 08:33:54.048611 2400 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:54.049731 2417 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74de1d5aa3e42f6a5839f42420d827c. Latest consensus state: current_term: 1 leader_uuid: "e74de1d5aa3e42f6a5839f42420d827c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } } }
I20260807 08:33:54.049765 2416 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e74de1d5aa3e42f6a5839f42420d827c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74de1d5aa3e42f6a5839f42420d827c" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42585 } } }
I20260807 08:33:54.049851 2417 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:54.049861 2416 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:54.050189 2429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:54.053117 2429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:54.058513 2429 catalog_manager.cc:1347] Generated new cluster ID: eede7a881c764959ac1a1dfc92c53802
I20260807 08:33:54.058563 2429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:54.084219 2429 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:54.084512 2429 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:54.096863 2429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74de1d5aa3e42f6a5839f42420d827c: Generated new TSK 0
I20260807 08:33:54.097581 2429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:54.107185 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2291
2026-08-07T08:33:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:54 dist-test-slave-6zbq krb5kdc[2447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:54 dist-test-slave-6zbq krb5kdc[2447](info): set up 2 sockets
Aug 07 08:33:54 dist-test-slave-6zbq krb5kdc[2447](info): commencing operation
krb5kdc: starting...
W20260807 08:33:55.227536 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:55.335282 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33103
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:55.438882 2467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:55.439102 2467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:55.439157 2467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:55.443014 2467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:55.443061 2467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:55.443087 2467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:55.443107 2467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:55.443135 2467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:55.448055 2467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:55.448897 2467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:55.449733 2467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:55.457247 2477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:55.457247 2475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:55.457351 2467 server_base.cc:1034] running on GCE node
W20260807 08:33:55.457228 2479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:55.458050 2467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:55.459049 2467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:55.460201 2467 hybrid_clock.cc:648] HybridClock initialized: now 1786091635460188 us; error 24 us; skew 500 ppm
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091635, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:55.462719 2467 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:55.463508 2467 webserver.cc:460] Webserver started at http://127.1.157.126:37123/ using document root <none> and password file <none>
I20260807 08:33:55.463986 2467 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:55.464046 2467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:55.464227 2467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:55.466058 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "2a0a6087aae8495eae5ca825a79265f6"
format_stamp: "Formatted at 2026-08-07 08:33:55 on dist-test-slave-6zbq"
server_key: "df6e0b04e15cc08e8ef34f7e65fb1acd"
server_key_iv: "2012bbfb2e9fd1c5c6cb0c1aacfe3032"
server_key_version: "encryptionkey@0"
I20260807 08:33:55.466580 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "2a0a6087aae8495eae5ca825a79265f6"
format_stamp: "Formatted at 2026-08-07 08:33:55 on dist-test-slave-6zbq"
server_key: "df6e0b04e15cc08e8ef34f7e65fb1acd"
server_key_iv: "2012bbfb2e9fd1c5c6cb0c1aacfe3032"
server_key_version: "encryptionkey@0"
I20260807 08:33:55.469972 2467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 08:33:55.472369 2487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:55.473469 2467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:55.473579 2467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "2a0a6087aae8495eae5ca825a79265f6"
format_stamp: "Formatted at 2026-08-07 08:33:55 on dist-test-slave-6zbq"
server_key: "df6e0b04e15cc08e8ef34f7e65fb1acd"
server_key_iv: "2012bbfb2e9fd1c5c6cb0c1aacfe3032"
server_key_version: "encryptionkey@0"
I20260807 08:33:55.473670 2467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:55.491969 2467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:55.494458 2467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:55.494612 2467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:55.500960 2467 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:33103
I20260807 08:33:55.500975 2568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:33103 every 8 connection(s)
I20260807 08:33:55.501858 2467 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:55.504359 2571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:55.509361 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: Bootstrap starting.
I20260807 08:33:55.510157 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2467
I20260807 08:33:55.510273 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:55.510541 1653 external_mini_cluster.cc:1428] Setting key f544212ecb76eaa4a4d965544fd130e7
I20260807 08:33:55.511905 2571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:55.512564 2571 log.cc:824] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:55.514569 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: No bootstrap required, opened a new log
I20260807 08:33:55.518535 2571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } }
I20260807 08:33:55.518769 2571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:55.518837 2571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a0a6087aae8495eae5ca825a79265f6, State: Initialized, Role: FOLLOWER
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:55.519232 2571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } }
I20260807 08:33:55.519358 2571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:55.519421 2571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:55.519512 2571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:55.520401 2571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } }
I20260807 08:33:55.520711 2571 leader_election.cc:302] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a0a6087aae8495eae5ca825a79265f6; no voters:
I20260807 08:33:55.521008 2571 leader_election.cc:288] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:55.521061 2579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:55.521283 2579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [term 1 LEADER]: Becoming Leader. State: Replica: 2a0a6087aae8495eae5ca825a79265f6, State: Running, Role: LEADER
I20260807 08:33:55.521597 2579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } }
I20260807 08:33:55.522181 2571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:55.523181 2584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a0a6087aae8495eae5ca825a79265f6. Latest consensus state: current_term: 1 leader_uuid: "2a0a6087aae8495eae5ca825a79265f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } } }
I20260807 08:33:55.523310 2584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:55.523659 2583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a0a6087aae8495eae5ca825a79265f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a0a6087aae8495eae5ca825a79265f6" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33103 } } }
I20260807 08:33:55.524367 2583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:55.524569 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:55.511581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37661 (local address 127.1.157.126:33103)
0807 08:33:55.511976 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:55.511994 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:55.512033 (+ 39us) server_negotiation.cc:407] Connection header received
0807 08:33:55.512607 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:55.512622 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:55.512990 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:55.513333 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:55.514148 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:55.515155 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:55.516310 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:55.516743 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:55.516854 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:55.519548 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:55.519552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:55.519563 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:55.519589 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:55.521136 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:55.521603 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:55.521610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:55.521616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:55.521763 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:55.522027 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:55.522030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:55.522032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:55.522476 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:55.522644 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:55.523431 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 08:33:55.523697 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
W20260807 08:33:55.527104 2605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:55.527168 2605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:55.527230 2594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:55.527485 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:55.527990 2594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:55.532967 2594 catalog_manager.cc:1347] Generated new cluster ID: 24fe0af89be14c59970816bf137c41de
I20260807 08:33:55.533041 2594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:55.536629 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:55.529242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44763 (local address 127.1.157.126:33103)
0807 08:33:55.529344 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:55.529348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:55.529448 (+ 100us) server_negotiation.cc:407] Connection header received
0807 08:33:55.529561 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:55.529564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:55.529602 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:55.529662 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:55.530241 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:55.530758 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:55.531622 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:55.531915 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:55.532014 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:55.533617 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:55.533621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:55.533623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:55.533646 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:55.535122 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:55.535574 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:55.535577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:55.535579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:55.535649 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:55.535905 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:55.535907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:55.535909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:55.536073 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:55.536174 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:55.536402 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 08:33:55.536497 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 08:33:55.551885 2594 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:55.553041 2594 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:55.558732 2594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a0a6087aae8495eae5ca825a79265f6: Generated new TSK 0
I20260807 08:33:55.559403 2594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:55.562774 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2467
2026-08-07T08:33:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2620](info): set up 2 sockets
Aug 07 08:33:55 dist-test-slave-6zbq krb5kdc[2620](info): commencing operation
krb5kdc: starting...
W20260807 08:33:56.679811 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:56 dist-test-slave-6zbq krb5kdc[2620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:56.792387 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:37187
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:33231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:37187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:56.897927 2641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:56.898154 2641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:56.898217 2641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:56.902077 2641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:56.902129 2641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:56.902155 2641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:56.902176 2641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:56.902211 2641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:56.908017 2641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:33231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:37187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:37187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:56.908914 2641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:56.909809 2641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:56.917505 2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:56.917580 2649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:56.917613 2641 server_base.cc:1034] running on GCE node
W20260807 08:33:56.917532 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:56.918255 2641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:56.919258 2641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:56.920413 2641 hybrid_clock.cc:648] HybridClock initialized: now 1786091636920401 us; error 24 us; skew 500 ppm
Aug 07 08:33:56 dist-test-slave-6zbq krb5kdc[2620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091636, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:56.928833 2641 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:56.929648 2641 webserver.cc:460] Webserver started at http://127.1.157.126:35775/ using document root <none> and password file <none>
I20260807 08:33:56.930146 2641 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:56.930210 2641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:56.930421 2641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:56.932331 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24"
format_stamp: "Formatted at 2026-08-07 08:33:56 on dist-test-slave-6zbq"
server_key: "70ee60cf505516c7d3bc540648fcb10f"
server_key_iv: "939c05ba1664a6ff8652407be779f52b"
server_key_version: "encryptionkey@0"
I20260807 08:33:56.932857 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24"
format_stamp: "Formatted at 2026-08-07 08:33:56 on dist-test-slave-6zbq"
server_key: "70ee60cf505516c7d3bc540648fcb10f"
server_key_iv: "939c05ba1664a6ff8652407be779f52b"
server_key_version: "encryptionkey@0"
I20260807 08:33:56.936372 2641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:33:56.938613 2663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:56.939765 2641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:33:56.939878 2641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24"
format_stamp: "Formatted at 2026-08-07 08:33:56 on dist-test-slave-6zbq"
server_key: "70ee60cf505516c7d3bc540648fcb10f"
server_key_iv: "939c05ba1664a6ff8652407be779f52b"
server_key_version: "encryptionkey@0"
I20260807 08:33:56.939970 2641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:56.967670 2641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:56.970095 2641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:56.970260 2641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:56.976778 2641 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:37187
I20260807 08:33:56.976783 2736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:37187 every 8 connection(s)
I20260807 08:33:56.977636 2641 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:56.980180 2737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:56.985186 2737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24: Bootstrap starting.
I20260807 08:33:56.987453 2737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:56.987537 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2641
I20260807 08:33:56.987679 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:56.987950 1653 external_mini_cluster.cc:1428] Setting key 5ac44ae57a7f3cedf9967e2c62d69b25
I20260807 08:33:56.988298 2737 log.cc:824] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:56.990067 2737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24: No bootstrap required, opened a new log
I20260807 08:33:56.993927 2737 raft_consensus.cc:348] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } }
I20260807 08:33:56.994158 2737 raft_consensus.cc:374] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:56.994221 2737 raft_consensus.cc:729] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68c7379fc1a4ab8948dfdd6c4b35b24, State: Initialized, Role: FOLLOWER
I20260807 08:33:56.994663 2737 consensus_queue.cc:260] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } }
I20260807 08:33:56.994804 2737 raft_consensus.cc:388] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:56.994871 2737 raft_consensus.cc:482] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:56.994967 2737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:56.995801 2737 raft_consensus.cc:504] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } }
I20260807 08:33:56.996002 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:56.988960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39083 (local address 127.1.157.126:37187)
0807 08:33:56.989343 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:56.989358 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:56.989391 (+ 33us) server_negotiation.cc:407] Connection header received
0807 08:33:56.989977 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:56.989992 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:56.990318 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:56.990606 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:56.990921 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:56.991855 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:56.993077 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:56.993504 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:56.993608 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:56.994444 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:56.994449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:56.994460 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:56.994810 (+ 350us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:56.994877 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:56.994905 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:56.994992 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:56.995132 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 08:33:56.995313 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":97,"threads_started":1}
I20260807 08:33:56.996120 2737 leader_election.cc:302] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a68c7379fc1a4ab8948dfdd6c4b35b24; no voters:
I20260807 08:33:56.996460 2737 leader_election.cc:288] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:56.996521 2742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:56.996722 2742 raft_consensus.cc:686] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [term 1 LEADER]: Becoming Leader. State: Replica: a68c7379fc1a4ab8948dfdd6c4b35b24, State: Running, Role: LEADER
I20260807 08:33:56.997053 2742 consensus_queue.cc:237] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } }
I20260807 08:33:56.997522 2737 sys_catalog.cc:564] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:56.998215 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68c7379fc1a4ab8948dfdd6c4b35b24. Latest consensus state: current_term: 1 leader_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } } }
I20260807 08:33:56.998425 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:56.998762 2745 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68c7379fc1a4ab8948dfdd6c4b35b24" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 37187 } } }
I20260807 08:33:56.998859 2745 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:56.998849 2758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:56.999536 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:57.002043 2758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:57.007364 2758 catalog_manager.cc:1347] Generated new cluster ID: 0520e2213f9545259667c3ceef64f57c
I20260807 08:33:57.007424 2758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:57.007774 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:57.001630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49101 (local address 127.1.157.126:37187)
0807 08:33:57.001745 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:57.001749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:57.003842 (+ 2093us) server_negotiation.cc:407] Connection header received
0807 08:33:57.003932 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:57.003935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:57.003981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:57.004059 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:57.004341 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:57.005106 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:57.006064 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:57.006470 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:57.006579 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:57.007303 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:57.007307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:57.007310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:57.007322 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:57.007351 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:57.007365 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:57.007438 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:57.007595 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 08:33:57.007647 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:33:57.037842 2758 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:57.039021 2758 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:57.048703 2758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a68c7379fc1a4ab8948dfdd6c4b35b24: Generated new TSK 0
I20260807 08:33:57.049414 2758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:57.056588 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2641
2026-08-07T08:33:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:57 dist-test-slave-6zbq krb5kdc[2787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:57 dist-test-slave-6zbq krb5kdc[2787](info): set up 2 sockets
Aug 07 08:33:57 dist-test-slave-6zbq krb5kdc[2787](info): commencing operation
krb5kdc: starting...
W20260807 08:33:58.175066 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:58 dist-test-slave-6zbq krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:58.289091 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43001
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:43001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:58.394855 2808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:58.395098 2808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:58.395155 2808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:58.399047 2808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:58.399098 2808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:58.399125 2808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:58.399147 2808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:58.399173 2808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:58.404193 2808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:43001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:43001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:58.405095 2808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:58.406002 2808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:58.413972 2824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:58.414084 2808 server_base.cc:1034] running on GCE node
W20260807 08:33:58.413982 2820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:58.413982 2818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:58.414877 2808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:58.415860 2808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:58.417019 2808 hybrid_clock.cc:648] HybridClock initialized: now 1786091638417005 us; error 25 us; skew 500 ppm
Aug 07 08:33:58 dist-test-slave-6zbq krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091638, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:58.419554 2808 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:58.420315 2808 webserver.cc:460] Webserver started at http://127.1.157.126:41951/ using document root <none> and password file <none>
I20260807 08:33:58.420774 2808 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:58.420828 2808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:58.420984 2808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:58.422892 2808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "fa67a69856bb4ba69f6f4960b60ec3f1"
format_stamp: "Formatted at 2026-08-07 08:33:58 on dist-test-slave-6zbq"
server_key: "58560eff8f288c7feebe848f5ac35dd9"
server_key_iv: "161ff9155762251aa0180046451e2d59"
server_key_version: "encryptionkey@0"
I20260807 08:33:58.423413 2808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "fa67a69856bb4ba69f6f4960b60ec3f1"
format_stamp: "Formatted at 2026-08-07 08:33:58 on dist-test-slave-6zbq"
server_key: "58560eff8f288c7feebe848f5ac35dd9"
server_key_iv: "161ff9155762251aa0180046451e2d59"
server_key_version: "encryptionkey@0"
I20260807 08:33:58.426734 2808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:33:58.428833 2836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:58.429880 2808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:33:58.429973 2808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "fa67a69856bb4ba69f6f4960b60ec3f1"
format_stamp: "Formatted at 2026-08-07 08:33:58 on dist-test-slave-6zbq"
server_key: "58560eff8f288c7feebe848f5ac35dd9"
server_key_iv: "161ff9155762251aa0180046451e2d59"
server_key_version: "encryptionkey@0"
I20260807 08:33:58.430048 2808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:58.434077 2808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:58.436494 2808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:58.436619 2808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:58.442855 2808 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:43001
I20260807 08:33:58.442878 2924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:43001 every 8 connection(s)
I20260807 08:33:58.443674 2808 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:58.444151 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2808
I20260807 08:33:58.444291 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:58.444540 1653 external_mini_cluster.cc:1428] Setting key 727c24d5a502a655c494aea570e977f3
I20260807 08:33:58.446616 2925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:58.449242 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:58.445800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55931 (local address 127.1.157.126:43001)
0807 08:33:58.446207 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:58.446221 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:58.446256 (+ 35us) server_negotiation.cc:407] Connection header received
0807 08:33:58.446933 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:58.446951 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:58.447213 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:58.447502 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:58.448202 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:58.448206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:58.448217 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:58.448227 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:58.448277 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:58.448296 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:58.448415 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:58.448485 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 08:33:58.448624 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":129,"threads_started":1}
I20260807 08:33:58.451649 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:33:58.452050 2925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: Bootstrap starting.
I20260807 08:33:58.454210 2925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: Neither blocks nor log segments found. Creating new log.
I20260807 08:33:58.454880 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:58.453438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39689 (local address 127.1.157.126:43001)
0807 08:33:58.453532 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:58.453536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:58.453707 (+ 171us) server_negotiation.cc:407] Connection header received
0807 08:33:58.453808 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:58.453810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:58.453840 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:58.453898 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:58.454538 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:58.454542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:58.454544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 08:33:58.454555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:58.454580 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 08:33:58.454590 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:58.454657 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:58.454731 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 08:33:58.454772 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 08:33:58.454985 2925 log.cc:824] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:33:58.456784 2925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: No bootstrap required, opened a new log
I20260807 08:33:58.460078 2925 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } }
I20260807 08:33:58.460285 2925 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 08:33:58.460341 2925 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa67a69856bb4ba69f6f4960b60ec3f1, State: Initialized, Role: FOLLOWER
I20260807 08:33:58.460702 2925 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } }
I20260807 08:33:58.460832 2925 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 08:33:58.460894 2925 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 08:33:58.460992 2925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:33:58.461848 2925 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } }
I20260807 08:33:58.462144 2925 leader_election.cc:302] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa67a69856bb4ba69f6f4960b60ec3f1; no voters:
I20260807 08:33:58.462445 2925 leader_election.cc:288] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 08:33:58.462538 2941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:33:58.462733 2941 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [term 1 LEADER]: Becoming Leader. State: Replica: fa67a69856bb4ba69f6f4960b60ec3f1, State: Running, Role: LEADER
I20260807 08:33:58.463048 2941 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } }
I20260807 08:33:58.463507 2925 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:33:58.464473 2942 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } } }
I20260807 08:33:58.464593 2942 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [sys.catalog]: This master's current role is: LEADER
I20260807 08:33:58.464954 2944 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa67a69856bb4ba69f6f4960b60ec3f1. Latest consensus state: current_term: 1 leader_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa67a69856bb4ba69f6f4960b60ec3f1" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 43001 } } }
I20260807 08:33:58.465044 2944 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1 [sys.catalog]: This master's current role is: LEADER
W20260807 08:33:58.467782 2968 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:33:58.467844 2968 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:33:58.467933 2959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:33:58.468827 2959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:33:58.473634 2959 catalog_manager.cc:1347] Generated new cluster ID: 018267b1c2cc49c99f4576509348be61
I20260807 08:33:58.473695 2959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:33:58.490624 2959 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:33:58.490876 2959 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:33:58.494050 2959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa67a69856bb4ba69f6f4960b60ec3f1: Generated new TSK 0
I20260807 08:33:58.494617 2959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 08:33:58.497983 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2808
2026-08-07T08:33:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1442 ms)
[----------] 18 tests from AuthTokenIssuingTest (26396 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:33:58 dist-test-slave-6zbq krb5kdc[2975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:33:58 dist-test-slave-6zbq krb5kdc[2975](info): set up 2 sockets
Aug 07 08:33:58 dist-test-slave-6zbq krb5kdc[2975](info): commencing operation
krb5kdc: starting...
W20260807 08:33:59.614615 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:33:59 dist-test-slave-6zbq krb5kdc[2975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:33:59.722612 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42169
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:33:59.826297 2993 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:33:59.826558 2993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:33:59.826598 2993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:33:59.826658 2993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:33:59.830498 2993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:33:59.830544 2993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:33:59.830570 2993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:33:59.830591 2993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:33:59.830616 2993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:33:59.835507 2993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:33:59.836383 2993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:33:59.837255 2993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:33:59.844866 3000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:59.844868 3002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:33:59.844897 3004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:33:59.845000 2993 server_base.cc:1034] running on GCE node
I20260807 08:33:59.845602 2993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:33:59.846632 2993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:33:59.847771 2993 hybrid_clock.cc:648] HybridClock initialized: now 1786091639847752 us; error 32 us; skew 500 ppm
Aug 07 08:33:59 dist-test-slave-6zbq krb5kdc[2975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:33:59.850165 2993 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:33:59.850950 2993 webserver.cc:460] Webserver started at http://127.1.157.126:45371/ using document root <none> and password file <none>
I20260807 08:33:59.851423 2993 fs_manager.cc:359] Metadata directory not provided
I20260807 08:33:59.851480 2993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:33:59.851657 2993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:33:59.853752 2993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "c547f86693a846eb8975ba6f1f829251"
format_stamp: "Formatted at 2026-08-07 08:33:59 on dist-test-slave-6zbq"
server_key: "d01fc41874754d82ccfe21013747ccfc"
server_key_iv: "e156a9ae7282bf4be53776f53c32be0d"
server_key_version: "encryptionkey@0"
I20260807 08:33:59.854249 2993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "c547f86693a846eb8975ba6f1f829251"
format_stamp: "Formatted at 2026-08-07 08:33:59 on dist-test-slave-6zbq"
server_key: "d01fc41874754d82ccfe21013747ccfc"
server_key_iv: "e156a9ae7282bf4be53776f53c32be0d"
server_key_version: "encryptionkey@0"
I20260807 08:33:59.857537 2993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 08:33:59.859659 3015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:33:59.860677 2993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 08:33:59.860783 2993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "c547f86693a846eb8975ba6f1f829251"
format_stamp: "Formatted at 2026-08-07 08:33:59 on dist-test-slave-6zbq"
server_key: "d01fc41874754d82ccfe21013747ccfc"
server_key_iv: "e156a9ae7282bf4be53776f53c32be0d"
server_key_version: "encryptionkey@0"
I20260807 08:33:59.860870 2993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:33:59.876904 2993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:33:59.879256 2993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:33:59.879415 2993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:33:59.885430 2993 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:42169
I20260807 08:33:59.885658 3086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:42169 every 8 connection(s)
I20260807 08:33:59.886224 2993 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:33:59.887233 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 2993
I20260807 08:33:59.887372 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:33:59.887646 1653 external_mini_cluster.cc:1428] Setting key fa35ee325e5f67a8e6d40b2b1d6de6d6
I20260807 08:33:59.889309 3087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:33:59.892833 3087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.157.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.125@KRBTEST.COM" created.
Aug 07 08:33:59 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:33:59.906647 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.894773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.1.157.126:42169)
0807 08:33:59.895283 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 08:33:59.895311 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 08:33:59.895356 (+ 45us) server_negotiation.cc:407] Connection header received
0807 08:33:59.896304 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:33:59.896309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:33:59.896470 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:33:59.896629 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:33:59.897319 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:59.898253 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:59.899578 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:33:59.899923 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:33:59.900039 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:59.902372 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:33:59.902379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:33:59.902392 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:33:59.902417 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:33:59.904076 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:59.904511 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:59.904517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:59.904519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:59.904653 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:33:59.904916 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:33:59.904919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:33:59.904921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:33:59.905369 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:33:59.905503 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 08:33:59.905802 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 08:33:59.906026 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":401,"spinlock_wait_cycles":6528,"thread_start_us":148,"threads_started":1}
I20260807 08:33:59.906646 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.894772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42169 (local address 127.0.0.1:40594)
0807 08:33:59.895192 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:59.895240 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 08:33:59.896033 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:33:59.896641 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:33:59.896647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:33:59.896965 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:33:59.897194 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:59.897207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:59.898403 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:59.898406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:59.899469 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:33:59.899481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:33:59.900071 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:33:59.900075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:33:59.900197 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:33:59.900711 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:33:59.900732 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:33:59.902193 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:33:59.904177 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:59.904185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:59.904192 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:59.904410 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:59.904736 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:33:59.904738 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:33:59.904739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:33:59.904835 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:33:59.905464 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:33:59.905468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:33:59.905576 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 08:33:59.905698 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 08:33:59.905933 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":100,"threads_started":1}
I20260807 08:33:59.908788 3087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:33:59.909372 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.909135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:57992)
0807 08:33:59.909247 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:59.909323 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:33:59.909525 3017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
I20260807 08:33:59.910292 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.910101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:57996)
0807 08:33:59.910189 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:59.910231 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:33:59.910436 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:42271: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
I20260807 08:33:59.913126 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:42271
--webserver_interface=127.1.157.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:33:59.959676 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } attempt: 1
I20260807 08:33:59.960494 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.960211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58010)
0807 08:33:59.960360 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:59.960418 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 08:33:59.961151 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:33:59.960940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58018)
0807 08:33:59.961038 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:33:59.961094 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 08:33:59.961270 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:42271: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
W20260807 08:34:00.019480 3094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:34:00.019791 3094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:00.019838 3094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:00.019902 3094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:00.023975 3094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:00.024044 3094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:00.024070 3094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:00.024092 3094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:00.024121 3094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:00.029417 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } attempt: 2
I20260807 08:34:00.029631 3094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.157.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:00.030218 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.029933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58024)
0807 08:34:00.030078 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.030143 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 08:34:00.030790 3094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:00.030931 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.030706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58030)
0807 08:34:00.030814 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.030861 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:34:00.031049 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:42271: Network error: Client connection negotiation failed: client connection to 127.1.157.125:42271: connect: Connection refused (error 111)
I20260807 08:34:00.031862 3094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:00.040139 3100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:00.040166 3101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:00.040320 3094 server_base.cc:1034] running on GCE node
W20260807 08:34:00.040146 3103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:00.040961 3094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:00.042258 3094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:00.043455 3094 hybrid_clock.cc:648] HybridClock initialized: now 1786091640043434 us; error 37 us; skew 500 ppm
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:00.046324 3094 init.cc:377] Logged in from keytab as kudu/127.1.157.125@KRBTEST.COM (short username kudu)
I20260807 08:34:00.047258 3094 webserver.cc:460] Webserver started at http://127.1.157.125:41743/ using document root <none> and password file <none>
I20260807 08:34:00.047796 3094 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:00.047878 3094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:00.048070 3094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:00.050128 3094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data/instance:
uuid: "a4880a499de64ae0bb601f567edbf5f1"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "b260f344e9459e8e499999c8cbe076eb"
server_key_iv: "338b3a10b573ff210d475e99b3cd68fe"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.050683 3094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal/instance:
uuid: "a4880a499de64ae0bb601f567edbf5f1"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "b260f344e9459e8e499999c8cbe076eb"
server_key_iv: "338b3a10b573ff210d475e99b3cd68fe"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.054271 3094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 08:34:00.056617 3112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:00.057753 3094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:34:00.057862 3094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "a4880a499de64ae0bb601f567edbf5f1"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "b260f344e9459e8e499999c8cbe076eb"
server_key_iv: "338b3a10b573ff210d475e99b3cd68fe"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.057966 3094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:00.075243 3094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:00.078006 3094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:00.078161 3094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:00.084872 3094 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.125:42271
I20260807 08:34:00.085131 3188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.125:42271 every 8 connection(s)
I20260807 08:34:00.085800 3094 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
I20260807 08:34:00.088466 3190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:00.088564 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3094
I20260807 08:34:00.088699 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-1/wal/instance
I20260807 08:34:00.088953 1653 external_mini_cluster.cc:1428] Setting key 984ad96ec36fb4a463b3b3e2e1ca5cc1
I20260807 08:34:00.092706 3190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.157.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.124@KRBTEST.COM" created.
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:34:00.106680 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.094913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.1.157.126:42169)
0807 08:34:00.095076 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.095081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.095880 (+ 799us) server_negotiation.cc:407] Connection header received
0807 08:34:00.096772 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.096776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.096844 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.096950 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.098051 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.098656 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.099895 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.100150 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.100245 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.102847 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.102850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.102852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.102877 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.104723 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.105194 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.105196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.105198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.105273 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.105587 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.105590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.105592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.105778 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.105865 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.106384 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.106512 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 08:34:00.107427 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.095219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42169 (local address 127.0.0.1:40600)
0807 08:34:00.095686 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.095770 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.096575 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.097072 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.097078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.097395 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.097908 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.097923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.098809 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.098812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.099766 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.099778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.100330 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.100335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.100540 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.101148 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.101168 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.102687 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.104837 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.104843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.104857 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.105100 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.105363 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.105366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.105368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.105500 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.105878 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.105884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.106122 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.106282 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.106615 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260807 08:34:00.108791 3190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
I20260807 08:34:00.115406 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:37643
--webserver_interface=127.1.157.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:41209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:00.117085 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.109244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58040)
0807 08:34:00.109400 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.109411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.109467 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.109985 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.109988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.110139 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.110218 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.110223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.111148 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.111151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.111674 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.111681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.112200 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.112204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.112361 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.112790 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.112802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.113930 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.115640 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.115641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.115643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.115806 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.116102 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.116104 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.116105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.116165 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.116605 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.116607 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.116694 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.116806 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.116923 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":49}
I20260807 08:34:00.117130 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.109390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.1.157.125:42271)
0807 08:34:00.109674 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.109682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.109699 (+ 17us) server_negotiation.cc:407] Connection header received
0807 08:34:00.109729 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.109733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.109876 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.109986 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.110316 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.111049 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.111772 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.112106 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.112194 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.114046 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.114055 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.114066 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.114092 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.115535 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.115911 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.115917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.115919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.116016 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.116231 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.116234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.116235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.116518 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.116650 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.116885 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.117003 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":209,"thread_start_us":118,"threads_started":1}
I20260807 08:34:00.118718 3190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:00.119361 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.119135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56152)
0807 08:34:00.119239 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.119311 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 08:34:00.119531 3116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
I20260807 08:34:00.120327 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.120131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56158)
0807 08:34:00.120232 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.120281 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 08:34:00.120442 3190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
I20260807 08:34:00.122181 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } attempt: 3
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:34:00.129539 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.122650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.1.157.125:42271)
0807 08:34:00.122742 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.122746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.122799 (+ 53us) server_negotiation.cc:407] Connection header received
0807 08:34:00.122925 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.122928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.122967 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.123028 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.123415 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.123916 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.124662 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.124913 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.124996 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.126950 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.126953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.126955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.126976 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.128189 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.128599 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.128601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.128602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.128666 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.128900 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.128902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.128903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.129050 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.129115 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.129351 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.129436 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:34:00.129594 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.122616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58050)
0807 08:34:00.122743 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.122757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.122847 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.123056 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.123059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.123220 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.123315 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.123321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.124013 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.124017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.124559 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.124567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.125011 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.125015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.125115 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.125515 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.125529 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.126826 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.128280 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.128283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.128285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.128521 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.128733 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.128736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.128738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.128828 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.129118 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.129121 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.129216 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.129308 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.129434 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:34:00.130626 3087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:00.131250 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.131045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56174)
0807 08:34:00.131150 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.131196 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:34:00.131846 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.131624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56176)
0807 08:34:00.131756 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.131798 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 08:34:00.131973 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
I20260807 08:34:00.163131 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 1
I20260807 08:34:00.163774 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.163547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56186)
0807 08:34:00.163660 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.163709 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:34:00.164450 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.164202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56202)
0807 08:34:00.164321 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.164371 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 08:34:00.164567 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
I20260807 08:34:00.169682 3190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 1
I20260807 08:34:00.170297 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.170053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56212)
0807 08:34:00.170185 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.170237 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 08:34:00.170995 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.170767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56216)
0807 08:34:00.170897 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.170943 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 08:34:00.171145 3190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
W20260807 08:34:00.223013 3205 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:34:00.223237 3205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:00.223259 3205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:00.223330 3205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:00.227186 3205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:00.227233 3205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:00.227252 3205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:00.227267 3205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:00.227280 3205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:00.232163 3205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:41209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:42169,127.1.157.125:42271,127.1.157.124:37643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:37643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.157.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:00.233006 3205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:00.233887 3205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:34:00.239295 3190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 2
I20260807 08:34:00.239711 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 2
I20260807 08:34:00.240013 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.239735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56218)
0807 08:34:00.239867 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.239926 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 08:34:00.240353 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.240091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56228)
0807 08:34:00.240211 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.240262 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 08:34:00.241190 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.240905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56244)
0807 08:34:00.241052 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.241126 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 08:34:00.241353 3190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
I20260807 08:34:00.241613 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.241378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56254)
0807 08:34:00.241499 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.241550 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 08:34:00.241734 3221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:00.241784 3087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:37643: Network error: Client connection negotiation failed: client connection to 127.1.157.124:37643: connect: Connection refused (error 111)
W20260807 08:34:00.241820 3219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:00.241745 3214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:00.242575 3205 server_base.cc:1034] running on GCE node
I20260807 08:34:00.243050 3205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:00.243989 3205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:00.245136 3205 hybrid_clock.cc:648] HybridClock initialized: now 1786091640245112 us; error 37 us; skew 500 ppm
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:00.247638 3205 init.cc:377] Logged in from keytab as kudu/127.1.157.124@KRBTEST.COM (short username kudu)
I20260807 08:34:00.248409 3205 webserver.cc:460] Webserver started at http://127.1.157.124:39289/ using document root <none> and password file <none>
I20260807 08:34:00.248890 3205 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:00.248951 3205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:00.249122 3205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:00.251039 3205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data/instance:
uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "3b9e76b90b03159b9f2d530c6cd45d9a"
server_key_iv: "a36b9dcd19b4fee97a93914f94e6b322"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.251528 3205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal/instance:
uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "3b9e76b90b03159b9f2d530c6cd45d9a"
server_key_iv: "a36b9dcd19b4fee97a93914f94e6b322"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.254844 3205 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 08:34:00.256925 3232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:00.258026 3205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 08:34:00.258139 3205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7"
format_stamp: "Formatted at 2026-08-07 08:34:00 on dist-test-slave-6zbq"
server_key: "3b9e76b90b03159b9f2d530c6cd45d9a"
server_key_iv: "a36b9dcd19b4fee97a93914f94e6b322"
server_key_version: "encryptionkey@0"
I20260807 08:34:00.258221 3205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:00.273453 3205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:00.275832 3205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:00.275983 3205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:00.282140 3205 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.124:37643
I20260807 08:34:00.282401 3309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.124:37643 every 8 connection(s)
I20260807 08:34:00.282975 3205 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
I20260807 08:34:00.285394 3310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:00.288782 3310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:00.291327 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3205
I20260807 08:34:00.291460 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786091364559477-1653-0/minicluster-data/master-2/wal/instance
I20260807 08:34:00.291756 1653 external_mini_cluster.cc:1428] Setting key 11b45c9321293fb1b507792646fe77b0
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:00.300942 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.290393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.1.157.126:42169)
0807 08:34:00.290509 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.290512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.291249 (+ 737us) server_negotiation.cc:407] Connection header received
0807 08:34:00.292277 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.292280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.292343 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.292417 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.293549 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.294043 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.295016 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.295231 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.295323 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.297576 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.297580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.297583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.297613 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.298973 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.299486 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.299490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.299491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.299572 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.299889 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.299892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.299894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.300091 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.300195 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.300690 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.300823 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:34:00.301867 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.290700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:42169 (local address 127.0.0.1:40602)
0807 08:34:00.291087 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.291140 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.292089 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.292585 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.292593 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.292977 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.293414 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.293427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.294141 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.294144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.294905 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.294915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.295357 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.295361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.295512 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.295979 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.295994 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.297425 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.299075 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.299085 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.299094 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.299392 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.299655 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.299658 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.299660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.299801 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.300203 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.300207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.300442 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.300628 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.300905 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260807 08:34:00.301910 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.292579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.1.157.124:37643)
0807 08:34:00.292891 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.292899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.292918 (+ 19us) server_negotiation.cc:407] Connection header received
0807 08:34:00.292951 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.292955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.293091 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.293222 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.293916 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.294781 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.295580 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.296032 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.296126 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.298661 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.298666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.298675 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.298695 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.300036 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.300438 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.300443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.300445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.300524 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.300780 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.300783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.300785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.301048 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.301253 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.301612 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.301727 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":234,"thread_start_us":154,"threads_started":1}
I20260807 08:34:00.303020 3310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:34:00.310748 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.303456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:42271 (local address 127.0.0.1:58058)
0807 08:34:00.303755 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.303768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.303881 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.304148 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.304150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.304267 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.304362 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.304369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.305124 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.305126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.305691 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.305700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.306117 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.306121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.306218 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.306677 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.306688 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.307834 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.309402 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.309405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.309407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.309574 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.309816 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.309819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.309821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.309891 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.310275 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.310278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.310390 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.310480 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.310591 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260807 08:34:00.310778 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.303589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.1.157.125:42271)
0807 08:34:00.303733 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.303737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.303820 (+ 83us) server_negotiation.cc:407] Connection header received
0807 08:34:00.304006 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.304009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.304054 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.304119 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.304512 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.305020 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.305812 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.306028 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.306096 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.307965 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.307969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.307972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.308003 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.309304 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.309659 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.309662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.309664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.309735 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.309969 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.309972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.309974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.310161 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.310252 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.310555 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.310650 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 08:34:00.311538 3310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:00.312134 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.303934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.1.157.126:42169)
0807 08:34:00.304247 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.304250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.304260 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:34:00.304287 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.304289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.304318 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.304373 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.304964 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.305483 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.306389 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.306705 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.306785 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.308888 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.308892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.308894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.308919 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.310150 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.310586 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.310589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.310591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.310673 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.310977 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.310981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.310982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.311177 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.311376 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.311751 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.311989 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:34:00.318691 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.312041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.1.157.124:37643)
0807 08:34:00.312143 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.312147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.312178 (+ 31us) server_negotiation.cc:407] Connection header received
0807 08:34:00.312269 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.312271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.312310 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.312381 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.312664 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.313210 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.314028 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.314287 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.314481 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.316086 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.316089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.316091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.316113 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.317325 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.317671 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.317673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.317675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.317738 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.317965 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.317968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.317970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.318113 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.318190 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.318470 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.318568 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":37}
I20260807 08:34:00.318709 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.311971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56262)
0807 08:34:00.312129 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.312140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.312203 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.312385 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.312387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.312498 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.312561 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.312566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.313449 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.313452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.313943 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.313951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.314419 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.314423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.314537 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.314900 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.314909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.315976 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.317416 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.317419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.317421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.317593 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.317819 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.317822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.317824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.317894 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.318194 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.318197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.318272 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.318401 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.318520 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":23}
I20260807 08:34:00.321640 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.313364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.1.157.125:42271)
0807 08:34:00.313454 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.313457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.314160 (+ 703us) server_negotiation.cc:407] Connection header received
0807 08:34:00.314262 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.314265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.314303 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.314415 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.315079 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.315620 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.316366 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.316673 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.316772 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.318863 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.318866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.318868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.318894 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.320201 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.320646 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.320648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.320649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.320712 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.320948 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.320950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.320951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.321097 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.321256 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.321421 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.321519 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 08:34:00.322100 3310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: Bootstrap starting.
I20260807 08:34:00.324419 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:00.324568 3310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:00.325244 3310 log.cc:824] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:00.325626 3051 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c547f86693a846eb8975ba6f1f829251"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40618
I20260807 08:34:00.327127 3310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: No bootstrap required, opened a new log
I20260807 08:34:00.329903 3310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.330111 3310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d67596f1ebb4ca4a2005dd4ee1da1a7, State: Initialized, Role: FOLLOWER
I20260807 08:34:00.330426 3310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.330843 3317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.330950 3317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.331667 3310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:00.332497 3190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 3
W20260807 08:34:00.336619 3341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:00.336692 3341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:00.335119 3205 master_runner.cc:186] Error getting master registration for 127.1.157.126:42169: OK, instance_id { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" instance_seqno: 1786091639883824 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:34:00.337852 3205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091640, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:00.340921 3087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } attempt: 3
I20260807 08:34:00.344028 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.332870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56266)
0807 08:34:00.332971 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.332984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.333057 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.333315 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.333334 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.333504 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.333594 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.333602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.334245 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.334248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.335000 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.335012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.337963 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.337967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.338053 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.338479 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.338493 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.339923 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.341873 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.341876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.341878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.342120 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.342460 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.342464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.342466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.342556 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.343021 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.343024 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.343339 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.343709 (+ 370us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.343867 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 08:34:00.344059 3314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.332924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56266 (local address 127.1.157.124:37643)
0807 08:34:00.333038 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.333042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.333057 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:34:00.333150 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.333154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.333217 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.333294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.333697 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.334151 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.335116 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.335416 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.335512 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.340057 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.340060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.340063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.340087 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.341744 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.342231 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.342235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.342237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.342333 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.342654 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.342658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.342659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.342885 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.342975 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.343795 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.343912 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 08:34:00 dist-test-slave-6zbq krb5kdc[2975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091639, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:00.347343 3190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: Bootstrap starting.
I20260807 08:34:00.350024 3190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:00.350275 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.341250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:37643 (local address 127.0.0.1:56268)
0807 08:34:00.341421 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:00.341435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:00.341512 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:00.341871 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:00.341874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:00.342025 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:00.342105 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.342110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.343180 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.343184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.343717 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:00.343726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.344872 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.344876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:00.344958 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.345375 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:00.345393 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:00.346805 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:00.348742 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.348745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.348747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.348998 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.349312 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:00.349315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:00.349317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:00.349419 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:00.349810 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:00.349814 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:00.349905 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 08:34:00.349992 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 08:34:00.350121 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 08:34:00.350713 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.341357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.1.157.124:37643)
0807 08:34:00.341666 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.341674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.341686 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:34:00.341717 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.341720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.341765 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.341842 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.342211 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.343073 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.344371 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.344742 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.344847 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.346946 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.346950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.346953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.346976 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.348645 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.349103 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.349106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.349108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.349188 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.349511 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.349514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.349516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.349721 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.349807 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.350056 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.350220 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260807 08:34:00.350908 3190 log.cc:824] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:00.352991 3190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: No bootstrap required, opened a new log
I20260807 08:34:00.353133 3087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: Bootstrap starting.
I20260807 08:34:00.355284 3087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:00.356069 3087 log.cc:824] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:00.356829 3190 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.357043 3190 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4880a499de64ae0bb601f567edbf5f1, State: Initialized, Role: FOLLOWER
I20260807 08:34:00.357363 3190 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.357903 3349 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.358084 3087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: No bootstrap required, opened a new log
I20260807 08:34:00.358210 3349 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.358624 3190 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:00.360913 3087 raft_consensus.cc:348] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.361186 3087 raft_consensus.cc:729] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c547f86693a846eb8975ba6f1f829251, State: Initialized, Role: FOLLOWER
I20260807 08:34:00.361482 3087 consensus_queue.cc:260] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.362771 3087 sys_catalog.cc:564] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 08:34:00.364284 3363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:00.364406 3363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:00.365219 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.365352 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.362143 3094 master_runner.cc:186] Error getting master registration for 127.1.157.126:42169: OK, instance_id { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" instance_seqno: 1786091639883824 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:34:00.366027 3094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:34:00.366161 3051 raft_consensus.cc:482] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:34:00.366273 3051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:00.367413 3051 raft_consensus.cc:504] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.368247 3051 leader_election.cc:288] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [CANDIDATE]: Term 1 election: Requested vote from peers a4880a499de64ae0bb601f567edbf5f1 (127.1.157.125:42271), 7d67596f1ebb4ca4a2005dd4ee1da1a7 (127.1.157.124:37643)
W20260807 08:34:00.368630 3386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:00.368709 3386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:00.369020 3147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c547f86693a846eb8975ba6f1f829251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a4880a499de64ae0bb601f567edbf5f1"
I20260807 08:34:00.369236 3147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:00.369186 3270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c547f86693a846eb8975ba6f1f829251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7"
I20260807 08:34:00.369442 3270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:00.370366 3147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c547f86693a846eb8975ba6f1f829251 in term 1.
I20260807 08:34:00.370714 3270 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c547f86693a846eb8975ba6f1f829251 in term 1.
I20260807 08:34:00.371222 3017 leader_election.cc:302] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d67596f1ebb4ca4a2005dd4ee1da1a7, c547f86693a846eb8975ba6f1f829251; no voters:
I20260807 08:34:00.371506 3365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:00.371712 3365 raft_consensus.cc:686] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [term 1 LEADER]: Becoming Leader. State: Replica: c547f86693a846eb8975ba6f1f829251, State: Running, Role: LEADER
I20260807 08:34:00.372017 3365 consensus_queue.cc:237] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } }
I20260807 08:34:00.373695 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c547f86693a846eb8975ba6f1f829251. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.373804 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:00.374306 3395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:00.375211 3395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:00.376824 2993 master_runner.cc:263] No leader master found from master c547f86693a846eb8975ba6f1f829251
I20260807 08:34:00.376895 2993 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 08:34:00.380941 3021 tablet.cc:2369] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:00.382443 3270 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [term 1 FOLLOWER]: Refusing update from remote peer c547f86693a846eb8975ba6f1f829251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:00.383096 3393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:00.386885 3317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c547f86693a846eb8975ba6f1f829251. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.387001 3317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.387928 3147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [term 1 FOLLOWER]: Refusing update from remote peer c547f86693a846eb8975ba6f1f829251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:00.388460 3393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:00.391598 3349 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c547f86693a846eb8975ba6f1f829251. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.391709 3349 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.395459 3393 sys_catalog.cc:455] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.395572 3393 sys_catalog.cc:458] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:00.396649 3349 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.396757 3349 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.397106 3396 mvcc.cc:205] Tried to move back new op lower bound from 7315831358996852736 to 7315831358967099392. Current Snapshot: MvccSnapshot[applied={T|T < 7315831358996852736}]
I20260807 08:34:00.400141 3406 mvcc.cc:205] Tried to move back new op lower bound from 7315831358996852736 to 7315831358967099392. Current Snapshot: MvccSnapshot[applied={T|T < 7315831358996852736}]
I20260807 08:34:00.401178 3365 sys_catalog.cc:455] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.401304 3365 sys_catalog.cc:458] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:00.401849 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.382747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.1.157.125:42271)
0807 08:34:00.382876 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.382880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.384075 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 08:34:00.384264 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.384268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.384315 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.384364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.394429 (+ 10065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.395100 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.396152 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.396445 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.396522 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.398318 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.398322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.398325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.398368 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.399954 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.400450 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.400453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.400455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.400542 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.400854 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.400858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.400859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.401075 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.401218 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.401580 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.401692 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 08:34:00.402311 3317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c547f86693a846eb8975ba6f1f829251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c547f86693a846eb8975ba6f1f829251" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 42169 } } peers { permanent_uuid: "a4880a499de64ae0bb601f567edbf5f1" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 42271 } } peers { permanent_uuid: "7d67596f1ebb4ca4a2005dd4ee1da1a7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 37643 } } }
I20260807 08:34:00.402451 3317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:00.404117 3404 mvcc.cc:205] Tried to move back new op lower bound from 7315831358996852736 to 7315831358967099392. Current Snapshot: MvccSnapshot[applied={T|T < 7315831358996852736}]
I20260807 08:34:00.410014 3395 catalog_manager.cc:1347] Generated new cluster ID: cf9c5e48c942483ab8dd04e415e6fe75
I20260807 08:34:00.410079 3395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:00.413035 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.383480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.1.157.124:37643)
0807 08:34:00.383585 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.383589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.391246 (+ 7657us) server_negotiation.cc:407] Connection header received
0807 08:34:00.391408 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.391412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.391455 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.391500 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.392418 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.393222 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.406432 (+ 13210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.406804 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.406878 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.408522 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.408526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.408529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.408554 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.410367 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.411691 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.411695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.411697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.411799 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.412087 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.412090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.412092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.412296 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.412393 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.412767 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.412885 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 08:34:00.416891 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:00.390652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.1.157.126:42169)
0807 08:34:00.390763 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.390767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.400475 (+ 9708us) server_negotiation.cc:407] Connection header received
0807 08:34:00.403127 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.403131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.403180 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.403230 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.404302 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.405040 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.408863 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:00.409191 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:00.409272 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:00.413505 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:00.413509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:00.413512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:00.413539 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:00.415144 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.415602 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.415606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.415607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.415690 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:00.415988 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:00.415992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:00.415994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:00.416190 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:00.416293 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:00.416630 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 08:34:00.416748 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:34:00.436647 3395 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:00.437805 3395 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:00.454886 3395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c547f86693a846eb8975ba6f1f829251: Generated new TSK 0
I20260807 08:34:00.455735 3395 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:34:00.488611 3091 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.485977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.1.157.126:42169)
0807 08:34:00.486096 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.486101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.487599 (+ 1498us) server_negotiation.cc:407] Connection header received
0807 08:34:00.487785 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.487788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.487836 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.487887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.488483 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40626: BlockingRecv error: recv got EOF from 127.0.0.1:40626 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 08:34:00.489878 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.486792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.1.157.125:42271)
0807 08:34:00.486906 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.486910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.489065 (+ 2155us) server_negotiation.cc:407] Connection header received
0807 08:34:00.489220 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.489223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.489277 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.489324 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.489768 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58074: BlockingRecv error: recv got EOF from 127.0.0.1:58074 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 08:34:00.491200 3346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.487099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.1.157.124:37643)
0807 08:34:00.487222 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.487227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.490236 (+ 3009us) server_negotiation.cc:407] Connection header received
0807 08:34:00.490547 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.490551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.490597 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.490646 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.491094 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56292: BlockingRecv error: recv got EOF from 127.0.0.1:56292 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 08:34:00.499877 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.498652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.1.157.125:42271)
0807 08:34:00.498746 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.498750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.498979 (+ 229us) server_negotiation.cc:407] Connection header received
0807 08:34:00.499177 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.499180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.499251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.499313 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.499788 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58082: BlockingRecv error: recv got EOF from 127.0.0.1:58082 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 08:34:00.507506 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.503363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.1.157.125:42271)
0807 08:34:00.503462 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.503466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.506619 (+ 3153us) server_negotiation.cc:407] Connection header received
0807 08:34:00.506795 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.506798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.506844 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.506889 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.507406 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58096: BlockingRecv error: recv got EOF from 127.0.0.1:58096 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 08:34:00.513000 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.511631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.1.157.125:42271)
0807 08:34:00.511723 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.511726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.512185 (+ 459us) server_negotiation.cc:407] Connection header received
0807 08:34:00.512368 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.512371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.512416 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.512461 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.512908 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58102: BlockingRecv error: recv got EOF from 127.0.0.1:58102 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 08:34:00.520736 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.519206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.1.157.125:42271)
0807 08:34:00.519326 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.519330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.519848 (+ 518us) server_negotiation.cc:407] Connection header received
0807 08:34:00.520031 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.520034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.520092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.520144 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.520646 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58106: BlockingRecv error: recv got EOF from 127.0.0.1:58106 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 08:34:00.532461 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.530926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.1.157.125:42271)
0807 08:34:00.531019 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.531022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.531549 (+ 527us) server_negotiation.cc:407] Connection header received
0807 08:34:00.531744 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.531748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.531792 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.531840 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.532367 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58118: BlockingRecv error: recv got EOF from 127.0.0.1:58118 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 08:34:00.552114 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.550672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.1.157.125:42271)
0807 08:34:00.550774 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.550778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.551274 (+ 496us) server_negotiation.cc:407] Connection header received
0807 08:34:00.551449 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.551452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.551497 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.551542 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.552006 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58126: BlockingRecv error: recv got EOF from 127.0.0.1:58126 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 08:34:00.587349 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.586269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.1.157.125:42271)
0807 08:34:00.586433 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.586437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.586535 (+ 98us) server_negotiation.cc:407] Connection header received
0807 08:34:00.586672 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.586676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.586724 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.586801 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.587237 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58142: BlockingRecv error: recv got EOF from 127.0.0.1:58142 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 08:34:00.654436 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.653359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.1.157.125:42271)
0807 08:34:00.653510 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.653513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.653616 (+ 103us) server_negotiation.cc:407] Connection header received
0807 08:34:00.653705 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.653708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.653760 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.653845 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.654289 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58148: BlockingRecv error: recv got EOF from 127.0.0.1:58148 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 08:34:00.785192 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:00.784245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.1.157.125:42271)
0807 08:34:00.784384 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:00.784387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:00.784485 (+ 98us) server_negotiation.cc:407] Connection header received
0807 08:34:00.784604 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:00.784607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:00.784653 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:00.784731 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:00.785093 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58156: BlockingRecv error: recv got EOF from 127.0.0.1:58156 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 08:34:01.044220 3204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:01.043270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.1.157.125:42271)
0807 08:34:01.043419 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.043423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.043512 (+ 89us) server_negotiation.cc:407] Connection header received
0807 08:34:01.043624 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.043627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.043674 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.043759 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:01.044110 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58160: BlockingRecv error: recv got EOF from 127.0.0.1:58160 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 08:34:01.337970 3341 catalog_manager.cc:1259] Loaded cluster ID: cf9c5e48c942483ab8dd04e415e6fe75
I20260807 08:34:01.338058 3341 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: loading cluster ID for follower catalog manager: success
I20260807 08:34:01.339753 3341 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: acquiring CA information for follower catalog manager: success
I20260807 08:34:01.340875 3341 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d67596f1ebb4ca4a2005dd4ee1da1a7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:34:01.365597 3363 catalog_manager.cc:1259] Loaded cluster ID: cf9c5e48c942483ab8dd04e415e6fe75
I20260807 08:34:01.365674 3363 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: loading cluster ID for follower catalog manager: success
I20260807 08:34:01.367282 3363 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: acquiring CA information for follower catalog manager: success
I20260807 08:34:01.368364 3363 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4880a499de64ae0bb601f567edbf5f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:34:01.562053 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:01.558330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.1.157.125:42271)
0807 08:34:01.558576 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.558579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.558595 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:34:01.558749 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.558752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.558890 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.558985 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:01.559245 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.559840 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.560668 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.560892 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.560965 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:01.561229 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:01.561727 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:01.561818 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:01.561880 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 08:34:01.561931 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260807 08:34:01.568341 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:01.565017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.1.157.124:37643)
0807 08:34:01.565241 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.565245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.565259 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:01.565386 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.565389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.565493 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.565586 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:01.565793 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.566283 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.567073 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.567287 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.567370 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:01.567634 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:01.567984 (+ 350us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:01.568081 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:01.568159 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 08:34:01.568211 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
I20260807 08:34:01.574446 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:01.570848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.1.157.125:42271)
0807 08:34:01.570951 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.570955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.571173 (+ 218us) server_negotiation.cc:407] Connection header received
0807 08:34:01.571357 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.571360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.571404 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.571446 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:01.571813 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.572410 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.573362 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.573630 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.573713 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:01.573966 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:01.574034 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:01.574073 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:01.574231 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 08:34:01.574286 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:34:01.575023 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:01.570872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.1.157.126:42169)
0807 08:34:01.571087 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.571091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.571106 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:34:01.571221 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.571224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.571367 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.571466 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:01.571842 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.572517 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.573465 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.573812 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.573894 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:01.574212 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:01.574647 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:01.574737 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:01.574831 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 08:34:01.574893 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260807 08:34:01.575115 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:01.571312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.1.157.124:37643)
0807 08:34:01.571409 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:01.571413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:01.571637 (+ 224us) server_negotiation.cc:407] Connection header received
0807 08:34:01.571721 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:01.571724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:01.571761 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:01.571822 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:01.572132 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.572810 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.573872 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:01.574160 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:01.574244 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:01.574675 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:01.574744 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:01.574789 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:01.574941 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 08:34:01.574991 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 08:34:01.577508 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 2993
I20260807 08:34:01.587895 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3094
I20260807 08:34:01.602151 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3205
2026-08-07T08:34:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3102 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 08:34:01 dist-test-slave-6zbq krb5kdc[3541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 08:34:01 dist-test-slave-6zbq krb5kdc[3541](info): set up 2 sockets
Aug 07 08:34:01 dist-test-slave-6zbq krb5kdc[3541](info): commencing operation
krb5kdc: starting...
W20260807 08:34:02.720970 1653 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/test-user.keytab.
Aug 07 08:34:02 dist-test-slave-6zbq krb5kdc[3541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T08:34:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T08:34:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.157.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
Entry for principal kudu/127.1.157.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab.
I20260807 08:34:02.841511 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33829
--webserver_interface=127.1.157.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 08:34:02.949343 3570 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:34:02.949599 3570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:02.949643 3570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:02.949702 3570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:02.953611 3570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:02.953662 3570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:02.953688 3570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:02.953709 3570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:02.953730 3570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:02.958704 3570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.126.keytab
--principal=kudu/127.1.157.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.126:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.157.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:02.959611 3570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:02.960525 3570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:02.968235 3577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:02.968252 3581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:02.968225 3576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:02.969537 3570 server_base.cc:1034] running on GCE node
I20260807 08:34:02.970052 3570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:02.971095 3570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:02.972257 3570 hybrid_clock.cc:648] HybridClock initialized: now 1786091642972240 us; error 31 us; skew 500 ppm
Aug 07 08:34:02 dist-test-slave-6zbq krb5kdc[3541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:02.977999 3570 init.cc:377] Logged in from keytab as kudu/127.1.157.126@KRBTEST.COM (short username kudu)
I20260807 08:34:02.978813 3570 webserver.cc:460] Webserver started at http://127.1.157.126:39781/ using document root <none> and password file <none>
I20260807 08:34:02.979282 3570 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:02.979346 3570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:02.979502 3570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:02.981382 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data/instance:
uuid: "0d1849e038944b13a03eb6f9dc6232a4"
format_stamp: "Formatted at 2026-08-07 08:34:02 on dist-test-slave-6zbq"
server_key: "cd6128ab06d16defdfe9d20795942c54"
server_key_iv: "56cf4d66deb89e60a01b7d453faceb0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:02.981870 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance:
uuid: "0d1849e038944b13a03eb6f9dc6232a4"
format_stamp: "Formatted at 2026-08-07 08:34:02 on dist-test-slave-6zbq"
server_key: "cd6128ab06d16defdfe9d20795942c54"
server_key_iv: "56cf4d66deb89e60a01b7d453faceb0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:02.985216 3570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 08:34:02.987366 3595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:02.988456 3570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 08:34:02.988556 3570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal
uuid: "0d1849e038944b13a03eb6f9dc6232a4"
format_stamp: "Formatted at 2026-08-07 08:34:02 on dist-test-slave-6zbq"
server_key: "cd6128ab06d16defdfe9d20795942c54"
server_key_iv: "56cf4d66deb89e60a01b7d453faceb0e"
server_key_version: "encryptionkey@0"
I20260807 08:34:02.988646 3570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:03.000479 3570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:03.003080 3570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:03.003232 3570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:03.009785 3570 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.126:33829
I20260807 08:34:03.010032 3673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.126:33829 every 8 connection(s)
I20260807 08:34:03.010710 3570 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/data/info.pb
I20260807 08:34:03.013301 3675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:03.016849 3675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:03.017715 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3570
I20260807 08:34:03.017813 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-0/wal/instance
I20260807 08:34:03.018071 1653 external_mini_cluster.cc:1428] Setting key e74b02812cfb47c5f5c3f82dbfbe067e
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.157.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.125@KRBTEST.COM" created.
I20260807 08:34:03.030983 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.019028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48002 (local address 127.1.157.126:33829)
0807 08:34:03.019412 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.019430 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.019685 (+ 255us) server_negotiation.cc:407] Connection header received
0807 08:34:03.020636 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.020640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.020818 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.020972 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.021813 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.023101 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.024108 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.024715 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.024887 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.026975 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.026982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.026995 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.027022 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.028484 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.028904 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.028908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.028909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.029001 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.029231 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.029233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.029234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.029542 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.029631 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.029982 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.030211 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":100,"threads_started":1}
I20260807 08:34:03.030988 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.019029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33829 (local address 127.0.0.1:48002)
0807 08:34:03.019540 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.019574 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.020356 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.020996 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.021002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.021434 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.021669 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.021682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.023220 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.023223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.023978 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.023988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.024852 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.024854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.024943 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.025443 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.025462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.026825 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.028589 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.028597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.028605 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.028819 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.029065 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.029067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.029068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.029168 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.029646 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.029651 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.029770 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.029946 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.030286 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":87,"spinlock_wait_cycles":4864,"thread_start_us":148,"threads_started":1}
I20260807 08:34:03.033409 3675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:03.034057 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.033806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51620)
0807 08:34:03.033918 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.033996 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 08:34:03.034258 3596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
I20260807 08:34:03.035024 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.034799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51622)
0807 08:34:03.034911 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.034963 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 08:34:03.035210 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:38421: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
Entry for principal kudu/127.1.157.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab.
I20260807 08:34:03.044661 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:38421
--webserver_interface=127.1.157.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:03.084470 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } attempt: 1
I20260807 08:34:03.085361 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.085038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51626)
0807 08:34:03.085198 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.085270 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 08:34:03.086024 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.085815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51634)
0807 08:34:03.085908 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.085957 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 08:34:03.086149 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:38421: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
W20260807 08:34:03.151319 3686 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:34:03.151556 3686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:03.151579 3686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:03.151636 3686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 08:34:03.154305 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } attempt: 2
I20260807 08:34:03.155092 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.154787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51650)
0807 08:34:03.154944 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.155007 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 08:34:03.155687 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.155493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51664)
0807 08:34:03.155576 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.155627 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 08:34:03.155769 3686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:03.155803 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.125:38421: Network error: Client connection negotiation failed: client connection to 127.1.157.125:38421: connect: Connection refused (error 111)
W20260807 08:34:03.155869 3686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:03.155905 3686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:03.155923 3686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:03.155951 3686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:03.161036 3686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.125.keytab
--principal=kudu/127.1.157.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.125:38421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.157.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:03.161924 3686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:03.162855 3686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 08:34:03.170408 3699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:03.170475 3697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:03.170397 3696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:03.170709 3686 server_base.cc:1034] running on GCE node
I20260807 08:34:03.171202 3686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:03.172109 3686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:03.173278 3686 hybrid_clock.cc:648] HybridClock initialized: now 1786091643173246 us; error 39 us; skew 500 ppm
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:03.175753 3686 init.cc:377] Logged in from keytab as kudu/127.1.157.125@KRBTEST.COM (short username kudu)
I20260807 08:34:03.176542 3686 webserver.cc:460] Webserver started at http://127.1.157.125:41469/ using document root <none> and password file <none>
I20260807 08:34:03.176988 3686 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:03.177040 3686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:03.177204 3686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:03.179092 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data/instance:
uuid: "bd8206b8355e4169906516c48bf93f3c"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "06fa144ebe2560a79b5d186b4b38720f"
server_key_iv: "b368f09fd54d930ef6fea140b8287cf4"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.179674 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal/instance:
uuid: "bd8206b8355e4169906516c48bf93f3c"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "06fa144ebe2560a79b5d186b4b38720f"
server_key_iv: "b368f09fd54d930ef6fea140b8287cf4"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.182950 3686 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 08:34:03.185014 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:03.186049 3686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 08:34:03.186168 3686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal
uuid: "bd8206b8355e4169906516c48bf93f3c"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "06fa144ebe2560a79b5d186b4b38720f"
server_key_iv: "b368f09fd54d930ef6fea140b8287cf4"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.186260 3686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:03.209139 3686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:03.211700 3686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:03.211860 3686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:03.218112 3686 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.125:38421
I20260807 08:34:03.218375 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.125:38421 every 8 connection(s)
I20260807 08:34:03.218992 3686 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/data/info.pb
I20260807 08:34:03.220284 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3686
I20260807 08:34:03.220374 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-1/wal/instance
I20260807 08:34:03.220641 1653 external_mini_cluster.cc:1428] Setting key 2cd03e64940f4a8db177324161125825
I20260807 08:34:03.221575 3798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:03.225497 3798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.157.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.157.124@KRBTEST.COM" created.
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:34:03.238018 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.227312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.1.157.126:33829)
0807 08:34:03.227456 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.227461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.228181 (+ 720us) server_negotiation.cc:407] Connection header received
0807 08:34:03.229034 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.229037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.229092 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.229186 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.230213 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.230782 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.232013 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.232258 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.232341 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.234684 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.234687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.234690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.234720 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.236169 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.236616 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.236619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.236621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.236714 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.236994 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.236997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.236999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.237220 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.237316 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.237724 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.237854 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 08:34:03.238647 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.227605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33829 (local address 127.0.0.1:48008)
0807 08:34:03.228032 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.228079 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.228841 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.229305 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.229311 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.229629 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.230057 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.230070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.230887 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.230889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.231898 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.231909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.232406 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.232410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.232602 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.233133 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.233153 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.234519 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.236276 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.236281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.236290 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.236517 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.236800 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.236802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.236804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.236914 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.237308 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.237312 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.237508 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.237643 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.237869 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260807 08:34:03.239879 3798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
Entry for principal kudu/127.1.157.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab.
I20260807 08:34:03.246511 1653 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
/tmp/dist-test-tasksV4IkV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:35075
--webserver_interface=127.1.157.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.157.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 08:34:03.246932 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } attempt: 3
I20260807 08:34:03.248063 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.240291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51676)
0807 08:34:03.240387 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.240398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.240453 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.240973 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.240976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.241139 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.241217 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.241222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.242087 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.242091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.242625 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.242633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.243135 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.243138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.243280 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.243632 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.243643 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.244699 (+ 1056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.246397 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.246401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.246403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.246623 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.246949 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.246952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.246954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.247096 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.247626 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.247628 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.247700 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.247794 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.247912 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":45}
I20260807 08:34:03.248651 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.240388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.1.157.125:38421)
0807 08:34:03.240682 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.240690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.240706 (+ 16us) server_negotiation.cc:407] Connection header received
0807 08:34:03.240736 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.240740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.240868 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.240975 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.241320 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.241989 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.242722 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.243039 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.243126 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.244811 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.244821 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.244830 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.244849 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.246268 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.246739 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.246744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.246745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.246839 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.247198 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.247200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.247201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.247469 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.247586 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.248337 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.248468 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":220,"thread_start_us":110,"threads_started":1}
I20260807 08:34:03.250278 3798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:03.250880 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.250646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46302)
0807 08:34:03.250739 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.250820 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 08:34:03.251364 3713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
I20260807 08:34:03.252458 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.251988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46308)
0807 08:34:03.252340 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.252395 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277}
W20260807 08:34:03.252712 3798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:34:03.256438 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.247334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51678)
0807 08:34:03.247495 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.247508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.247575 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.248014 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.248017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.248187 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.248314 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.248322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.249097 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.249101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.250327 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.250358 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.251783 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.251786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.251883 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.252283 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.252304 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.253417 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.255214 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.255216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.255218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.255405 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.255671 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.255673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.255674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.255734 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.256095 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.256096 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.256164 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.256231 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.256331 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 08:34:03.256562 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.247392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.1.157.125:38421)
0807 08:34:03.247789 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.247809 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.247822 (+ 13us) server_negotiation.cc:407] Connection header received
0807 08:34:03.247858 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.247860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.247898 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.247976 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.248424 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.248999 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.250457 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.250845 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.250951 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.253560 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.253564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.253566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.253592 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.255115 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.255491 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.255495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.255497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.255585 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.255806 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.255809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.255811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.256012 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.256109 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.256303 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.256410 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":78,"threads_started":1}
I20260807 08:34:03.257231 3675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } has no permanent_uuid. Determining permanent_uuid...
I20260807 08:34:03.257840 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.257640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46316)
0807 08:34:03.257741 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.257785 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 08:34:03.258476 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.258244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46324)
0807 08:34:03.258394 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.258432 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 08:34:03.258579 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
I20260807 08:34:03.289707 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 1
I20260807 08:34:03.290433 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.290158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46338)
0807 08:34:03.290282 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.290363 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 08:34:03.291011 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.290848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46346)
0807 08:34:03.290925 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.290966 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 08:34:03.291122 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
I20260807 08:34:03.301947 3798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 1
I20260807 08:34:03.302656 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.302383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46348)
0807 08:34:03.302511 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.302571 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 08:34:03.303304 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.303097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46352)
0807 08:34:03.303192 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.303242 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 08:34:03.303421 3798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
W20260807 08:34:03.356171 3805 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 08:34:03.356411 3805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 08:34:03.356434 3805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 08:34:03.356491 3805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 08:34:03.360374 3805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 08:34:03.360420 3805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 08:34:03.360438 3805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 08:34:03.360456 3805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 08:34:03.360484 3805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 08:34:03.365494 3805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.157.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.157.126:33829,127.1.157.125:38421,127.1.157.124:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/krb5kdc/kudu_127.1.157.124.keytab
--principal=kudu/127.1.157.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.157.124:35075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.157.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 07:56:15 UTC on ccycloud.quasar-eqkmoh.root.comops.site
build id 4572005
I20260807 08:34:03.366287 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 2
I20260807 08:34:03.366427 3805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 08:34:03.367121 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.366797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46362)
0807 08:34:03.366962 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.367033 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 08:34:03.367515 3805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 08:34:03.367821 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.367597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46366)
0807 08:34:03.367703 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.367757 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 08:34:03.367986 3675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
I20260807 08:34:03.371551 3798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 2
I20260807 08:34:03.372174 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.371934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46376)
0807 08:34:03.372054 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.372111 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 08:34:03.372807 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.372641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46384)
0807 08:34:03.372712 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.372752 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 08:34:03.372913 3798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.157.124:35075: Network error: Client connection negotiation failed: client connection to 127.1.157.124:35075: connect: Connection refused (error 111)
W20260807 08:34:03.375490 3812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:03.375494 3816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 08:34:03.375466 3813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 08:34:03.375694 3805 server_base.cc:1034] running on GCE node
I20260807 08:34:03.376291 3805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 08:34:03.377286 3805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 08:34:03.378403 3805 hybrid_clock.cc:648] HybridClock initialized: now 1786091643378387 us; error 27 us; skew 500 ppm
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 08:34:03.380946 3805 init.cc:377] Logged in from keytab as kudu/127.1.157.124@KRBTEST.COM (short username kudu)
I20260807 08:34:03.381804 3805 webserver.cc:460] Webserver started at http://127.1.157.124:44949/ using document root <none> and password file <none>
I20260807 08:34:03.382324 3805 fs_manager.cc:359] Metadata directory not provided
I20260807 08:34:03.382429 3805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 08:34:03.382645 3805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 08:34:03.384614 3805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data/instance:
uuid: "09caade5fe124056b05b38d737631fb7"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "c7f8dc5a1f09cef370a175626617d130"
server_key_iv: "53b16de6b6d11553a588b9ac37ca2258"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.385156 3805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal/instance:
uuid: "09caade5fe124056b05b38d737631fb7"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "c7f8dc5a1f09cef370a175626617d130"
server_key_iv: "53b16de6b6d11553a588b9ac37ca2258"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.388687 3805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 08:34:03.391099 3828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 08:34:03.392263 3805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 08:34:03.392372 3805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data,/tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal
uuid: "09caade5fe124056b05b38d737631fb7"
format_stamp: "Formatted at 2026-08-07 08:34:03 on dist-test-slave-6zbq"
server_key: "c7f8dc5a1f09cef370a175626617d130"
server_key_iv: "53b16de6b6d11553a588b9ac37ca2258"
server_key_version: "encryptionkey@0"
I20260807 08:34:03.392463 3805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 08:34:03.411783 3805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 08:34:03.414373 3805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 08:34:03.414525 3805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 08:34:03.420830 3805 rpc_server.cc:307] RPC server started. Bound to: 127.1.157.124:35075
I20260807 08:34:03.421087 3915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.157.124:35075 every 8 connection(s)
I20260807 08:34:03.421743 3805 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/data/info.pb
I20260807 08:34:03.423108 1653 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu as pid 3805
I20260807 08:34:03.423219 1653 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasksV4IkV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786091364559477-1653-0/minicluster-data/master-2/wal/instance
I20260807 08:34:03.423492 1653 external_mini_cluster.cc:1428] Setting key edd2f6703523e4d95a8b5f484c3dfb1a
I20260807 08:34:03.424733 3916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 08:34:03.429222 3916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
I20260807 08:34:03.437945 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.424729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.1.157.124:35075)
0807 08:34:03.425128 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.425156 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.425199 (+ 43us) server_negotiation.cc:407] Connection header received
0807 08:34:03.426063 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.426086 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.426515 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.426868 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.427470 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.428923 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.429686 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.430336 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.430511 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.433287 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.433294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.433305 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.433337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.435016 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.435435 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.435441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.435446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.435567 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.435803 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.435806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.435807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.436143 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.436305 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.436741 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.437006 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":123,"threads_started":1}
I20260807 08:34:03.439746 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.431046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.1.157.126:33829)
0807 08:34:03.431173 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.431177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.431455 (+ 278us) server_negotiation.cc:407] Connection header received
0807 08:34:03.431567 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.431570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.431638 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.431727 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.432432 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.433169 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.434274 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.434551 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.434632 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.436710 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.436714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.436716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.436741 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.438117 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.438579 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.438582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.438584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.438684 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.438918 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.438922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.438924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.439106 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.439193 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.439501 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.439602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 08:34:03.439769 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.430994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.126:33829 (local address 127.0.0.1:48016)
0807 08:34:03.431330 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.431376 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.431486 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.431764 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.431768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.432178 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.432303 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.432309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.433272 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.433276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.434163 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.434174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.434656 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.434660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.434766 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.435183 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.435194 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.436574 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.438218 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.438226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.438230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.438495 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.438761 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.438763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.438764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.438831 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.439206 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.439212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.439321 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.439432 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.439594 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260807 08:34:03.440834 3916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.126@KRBTEST.COM
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:34:03.448740 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.440162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48032 (local address 127.1.157.126:33829)
0807 08:34:03.440253 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.440256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.440266 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:34:03.440358 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.440360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.440399 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.440462 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.441062 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.441810 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.442586 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.442792 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.442870 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.444991 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.444994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.444995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.445015 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.446714 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.447188 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.447192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.447193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.447344 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.447665 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.447695 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.447698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.447961 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.448207 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.448490 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.448598 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 08:34:03.449508 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.441341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.1.157.125:38421)
0807 08:34:03.441455 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.441459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.441471 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:34:03.441512 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.441515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.441558 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.441637 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.441964 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.442536 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.443225 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.443481 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.443567 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.445458 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.445462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.445464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.445490 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.447600 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.448033 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.448036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.448038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.448121 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.448417 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.448420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.448422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.448625 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.448692 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.449270 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.449402 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:34:03.449879 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.441223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.125:38421 (local address 127.0.0.1:51688)
0807 08:34:03.441321 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.441333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.441392 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.441641 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.441643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.441756 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.441843 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.441847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.442640 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.442643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.443120 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.443127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.443577 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.443580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.443662 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.444017 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.444026 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.445335 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.447713 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.447717 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.447719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.447945 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.448203 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.448206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.448208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.448303 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.448763 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.448766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.449083 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.449683 (+ 600us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.449783 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 08:34:03.450584 3916 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.157.125@KRBTEST.COM
I20260807 08:34:03.459138 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.450836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.1.157.125:38421)
0807 08:34:03.450953 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.450957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.450972 (+ 15us) server_negotiation.cc:407] Connection header received
0807 08:34:03.451005 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.451008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.451046 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.451132 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.451777 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.452502 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.453332 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.453575 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.453658 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.456020 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.456023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.456026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.456050 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.457289 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.457647 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.457651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.457652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.457725 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.457964 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.457967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.457969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.458125 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.458319 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.458905 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.459002 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.124@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:03.463510 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.454951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46398)
0807 08:34:03.455049 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.455061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.455132 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.455381 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.455384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.455534 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.455642 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.455650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.456814 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.456817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.457746 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.457756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.458376 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.458379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.458449 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.458848 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.458876 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.460254 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.461865 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.461868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.461869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.462128 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.462460 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.462463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.462465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.462548 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.463013 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.463016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.463124 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.463223 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.463353 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 08:34:03.463580 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.455054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46398 (local address 127.1.157.124:35075)
0807 08:34:03.455172 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.455177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.455191 (+ 14us) server_negotiation.cc:407] Connection header received
0807 08:34:03.455226 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.455230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.455273 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.455358 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.455759 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.456689 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.457856 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.458168 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.458253 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.460384 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.460387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.460390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.460414 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.461759 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.462232 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.462234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.462236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.462306 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.462632 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.462635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.462636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.462813 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.462898 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.463295 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.463411 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":46}
I20260807 08:34:03.463863 1653 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 08:34:03.464064 3798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 3
I20260807 08:34:03.465067 3638 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0d1849e038944b13a03eb6f9dc6232a4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48032
I20260807 08:34:03.467152 3675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } attempt: 3
I20260807 08:34:03.467738 3916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: Bootstrap starting.
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091643, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.125@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:03.471093 3916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:03.472108 3916 log.cc:824] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:03.474575 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.464425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46400)
0807 08:34:03.464575 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.464587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.464655 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.465050 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.465053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.465214 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.465316 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.465322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.466285 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.466288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.467056 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.467067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.467696 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.467699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.467766 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.468810 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.468824 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.470167 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.472997 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.473001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.473003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.473231 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.473554 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.473557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.473559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.473644 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.474060 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.474063 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.474152 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.474230 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.474420 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
Aug 07 08:34:03 dist-test-slave-6zbq krb5kdc[3541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786091642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.157.126@KRBTEST.COM for kudu/127.1.157.124@KRBTEST.COM
I20260807 08:34:03.475046 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.464751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.1.157.124:35075)
0807 08:34:03.464850 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.464854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.464864 (+ 10us) server_negotiation.cc:407] Connection header received
0807 08:34:03.464893 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.464896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.464932 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.465420 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.465500 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.466161 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.467366 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.467603 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.467926 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.470317 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.470321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.470323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.470369 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.472890 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.473363 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.473366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.473368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.473447 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.473753 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.473755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.473757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.473949 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.474744 (+ 795us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.474818 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.474915 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 08:34:03.476125 3916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: No bootstrap required, opened a new log
I20260807 08:34:03.478075 3798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: Bootstrap starting.
I20260807 08:34:03.478611 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.468238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.157.124:35075 (local address 127.0.0.1:46406)
0807 08:34:03.468358 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 08:34:03.468372 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 08:34:03.468450 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 08:34:03.470751 (+ 2301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 08:34:03.470754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 08:34:03.470917 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 08:34:03.471025 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.471033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.472069 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.472073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.472744 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 08:34:03.472754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.473315 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.473319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 08:34:03.473405 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.473917 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 08:34:03.473932 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 08:34:03.475245 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 08:34:03.477072 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.477075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.477077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.477302 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.477606 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 08:34:03.477609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 08:34:03.477611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 08:34:03.477696 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 08:34:03.478141 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 08:34:03.478144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 08:34:03.478238 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 08:34:03.478315 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 08:34:03.478467 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 08:34:03.479210 3924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.468380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.1.157.124:35075)
0807 08:34:03.468669 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.468705 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.468717 (+ 12us) server_negotiation.cc:407] Connection header received
0807 08:34:03.468749 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.468752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.470624 (+ 1872us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.470712 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.471160 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.471971 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.472858 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.473183 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.473582 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.475375 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.475379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.475381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.475405 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.476948 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.477423 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.477425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.477427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.477506 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.477814 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.477817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.477819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.478025 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.478865 (+ 840us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.478938 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.479043 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":92,"threads_started":1}
I20260807 08:34:03.480495 3798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:03.481328 3798 log.cc:824] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:03.481158 3916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.481549 3916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09caade5fe124056b05b38d737631fb7, State: Initialized, Role: FOLLOWER
I20260807 08:34:03.481909 3916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.482682 3930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.482811 3930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.482936 3675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: Bootstrap starting.
I20260807 08:34:03.483867 3798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: No bootstrap required, opened a new log
I20260807 08:34:03.484160 3916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:03.485322 3675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: Neither blocks nor log segments found. Creating new log.
I20260807 08:34:03.486127 3675 log.cc:824] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: Log is configured to *not* fsync() on all Append() calls
I20260807 08:34:03.487819 3798 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.488128 3798 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8206b8355e4169906516c48bf93f3c, State: Initialized, Role: FOLLOWER
I20260807 08:34:03.488492 3798 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.488756 3675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: No bootstrap required, opened a new log
I20260807 08:34:03.489205 3947 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.489418 3947 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.490118 3798 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:03.488046 3805 master_runner.cc:186] Error getting master registration for 127.1.157.126:33829: OK, instance_id { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" instance_seqno: 1786091643007984 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:34:03.491736 3805 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:34:03.491632 3675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.491848 3675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1849e038944b13a03eb6f9dc6232a4, State: Initialized, Role: FOLLOWER
I20260807 08:34:03.492151 3675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.493315 3675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 08:34:03.493363 3638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 08:34:03.493458 3638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:03.494609 3638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.495945 3876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d1849e038944b13a03eb6f9dc6232a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "09caade5fe124056b05b38d737631fb7"
I20260807 08:34:03.495947 3755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d1849e038944b13a03eb6f9dc6232a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bd8206b8355e4169906516c48bf93f3c"
I20260807 08:34:03.496114 3876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:03.496133 3755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 08:34:03.494076 3686 master_runner.cc:186] Error getting master registration for 127.1.157.126:33829: OK, instance_id { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" instance_seqno: 1786091643007984 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 08:34:03.497083 3876 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d1849e038944b13a03eb6f9dc6232a4 in term 1.
I20260807 08:34:03.497133 3686 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 08:34:03.497313 3755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d1849e038944b13a03eb6f9dc6232a4 in term 1.
I20260807 08:34:03.495515 3638 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [CANDIDATE]: Term 1 election: Requested vote from peers bd8206b8355e4169906516c48bf93f3c (127.1.157.125:38421), 09caade5fe124056b05b38d737631fb7 (127.1.157.124:35075)
I20260807 08:34:03.497476 3600 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09caade5fe124056b05b38d737631fb7, 0d1849e038944b13a03eb6f9dc6232a4; no voters:
I20260807 08:34:03.498389 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.498524 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.498626 3985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 08:34:03.498701 3985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [term 1 LEADER]: Becoming Leader. State: Replica: 0d1849e038944b13a03eb6f9dc6232a4, State: Running, Role: LEADER
I20260807 08:34:03.498916 3985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } }
I20260807 08:34:03.500581 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1849e038944b13a03eb6f9dc6232a4. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.500700 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: This master's current role is: LEADER
W20260807 08:34:03.502503 3944 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:03.502576 3944 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 08:34:03.502787 3967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:03.502902 3967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 08:34:03.504834 3603 tablet.cc:2369] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 08:34:03.511368 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.500280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.1.157.125:38421)
0807 08:34:03.500385 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.500390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.501917 (+ 1527us) server_negotiation.cc:407] Connection header received
0807 08:34:03.502045 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.502048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.502096 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.502179 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.503033 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.503781 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.504838 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.505129 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.505229 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.508059 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.508063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.508065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.508090 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.509694 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.510140 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.510143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.510145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.510232 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.510535 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.510538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.510540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.510741 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.510862 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.511118 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.511227 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 08:34:03.513211 3984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 08:34:03.513281 3984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 08:34:03.513824 3994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 08:34:03.515398 3994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 08:34:03.519441 3924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.504089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.1.157.124:35075)
0807 08:34:03.504190 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.504193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.511412 (+ 7219us) server_negotiation.cc:407] Connection header received
0807 08:34:03.511658 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.511661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.511708 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.511788 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.512493 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.513200 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.514252 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.514661 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.514768 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.516112 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.516116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.516118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.516142 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.517731 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.518272 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.518275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.518277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.518393 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.518645 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.518648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.518649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.518850 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.518985 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.519182 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.519305 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 08:34:03.519608 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:03.501381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.1.157.126:33829)
0807 08:34:03.501479 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.501483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.504478 (+ 2995us) server_negotiation.cc:407] Connection header received
0807 08:34:03.505418 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.505421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.505461 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.505509 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.510421 (+ 4912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.511091 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.513590 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:03.513893 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:03.513990 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:03.515873 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 08:34:03.515877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 08:34:03.515879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 08:34:03.515903 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 08:34:03.517745 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.518325 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.518328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.518330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.518441 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 08:34:03.518748 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 08:34:03.518751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 08:34:03.518753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 08:34:03.518955 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 08:34:03.519047 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:03.519374 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 08:34:03.519479 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 08:34:03.526077 3755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [term 1 FOLLOWER]: Refusing update from remote peer 0d1849e038944b13a03eb6f9dc6232a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:03.527174 3876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [term 1 FOLLOWER]: Refusing update from remote peer 0d1849e038944b13a03eb6f9dc6232a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 08:34:03.527582 3959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:03.528471 3959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 08:34:03.534446 3947 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1849e038944b13a03eb6f9dc6232a4. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.534559 3947 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.536523 3930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1849e038944b13a03eb6f9dc6232a4. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.536634 3930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.545897 3959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.546016 3959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:03.546128 3959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.546214 3959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4 [sys.catalog]: This master's current role is: LEADER
I20260807 08:34:03.546212 3930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.546299 3930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.546350 4002 mvcc.cc:205] Tried to move back new op lower bound from 7315831371876605952 to 7315831371774791680. Current Snapshot: MvccSnapshot[applied={T|T < 7315831371876605952}]
I20260807 08:34:03.547027 4005 mvcc.cc:205] Tried to move back new op lower bound from 7315831371876605952 to 7315831371774791680. Current Snapshot: MvccSnapshot[applied={T|T < 7315831371876605952}]
I20260807 08:34:03.547538 3947 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d1849e038944b13a03eb6f9dc6232a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d1849e038944b13a03eb6f9dc6232a4" member_type: VOTER last_known_addr { host: "127.1.157.126" port: 33829 } } peers { permanent_uuid: "bd8206b8355e4169906516c48bf93f3c" member_type: VOTER last_known_addr { host: "127.1.157.125" port: 38421 } } peers { permanent_uuid: "09caade5fe124056b05b38d737631fb7" member_type: VOTER last_known_addr { host: "127.1.157.124" port: 35075 } } }
I20260807 08:34:03.547640 3947 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 08:34:03.548101 4003 mvcc.cc:205] Tried to move back new op lower bound from 7315831371876605952 to 7315831371774791680. Current Snapshot: MvccSnapshot[applied={T|T < 7315831371876605952}]
I20260807 08:34:03.559044 3994 catalog_manager.cc:1347] Generated new cluster ID: 660f32a204d04232b6d0736e30e400c7
I20260807 08:34:03.559109 3994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 08:34:03.580209 3994 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 08:34:03.581399 3994 catalog_manager.cc:1504] Loading token signing keys...
I20260807 08:34:03.601207 3994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d1849e038944b13a03eb6f9dc6232a4: Generated new TSK 0
I20260807 08:34:03.601890 3994 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 08:34:03.627673 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.624890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.1.157.125:38421)
0807 08:34:03.624993 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.624997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.626721 (+ 1724us) server_negotiation.cc:407] Connection header received
0807 08:34:03.626897 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.626901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.626944 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.626991 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.627558 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51714: BlockingRecv error: recv got EOF from 127.0.0.1:51714 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 08:34:03.628926 3678 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.625290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.1.157.126:33829)
0807 08:34:03.625389 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.625394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.628114 (+ 2720us) server_negotiation.cc:407] Connection header received
0807 08:34:03.628277 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.628280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.628323 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.628369 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.628823 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48044: BlockingRecv error: recv got EOF from 127.0.0.1:48044 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 08:34:03.630141 3924 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.625645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.1.157.124:35075)
0807 08:34:03.625744 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.625747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.629358 (+ 3611us) server_negotiation.cc:407] Connection header received
0807 08:34:03.629510 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.629513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.629556 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.629601 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.630036 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46422: BlockingRecv error: recv got EOF from 127.0.0.1:46422 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 08:34:03.634037 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.632614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.1.157.125:38421)
0807 08:34:03.632703 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.632707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.633178 (+ 471us) server_negotiation.cc:407] Connection header received
0807 08:34:03.633383 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.633387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.633450 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.633495 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.633948 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51728: BlockingRecv error: recv got EOF from 127.0.0.1:51728 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 08:34:03.638620 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.637222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.1.157.125:38421)
0807 08:34:03.637320 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.637324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.637826 (+ 502us) server_negotiation.cc:407] Connection header received
0807 08:34:03.637996 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.637999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.638039 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.638081 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.638540 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51738: BlockingRecv error: recv got EOF from 127.0.0.1:51738 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 08:34:03.643791 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.642424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.1.157.125:38421)
0807 08:34:03.642507 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.642511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.643007 (+ 496us) server_negotiation.cc:407] Connection header received
0807 08:34:03.643181 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.643184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.643225 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.643283 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.643712 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51754: BlockingRecv error: recv got EOF from 127.0.0.1:51754 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 08:34:03.651099 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.649620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.1.157.125:38421)
0807 08:34:03.649733 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.649736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.650275 (+ 539us) server_negotiation.cc:407] Connection header received
0807 08:34:03.650483 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.650486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.650528 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.650573 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.651022 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51766: BlockingRecv error: recv got EOF from 127.0.0.1:51766 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 08:34:03.662559 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.661098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.1.157.125:38421)
0807 08:34:03.661195 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.661199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.661640 (+ 441us) server_negotiation.cc:407] Connection header received
0807 08:34:03.661821 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.661824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.661868 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.661911 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.662474 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51774: BlockingRecv error: recv got EOF from 127.0.0.1:51774 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 08:34:03.682046 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.680682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.1.157.125:38421)
0807 08:34:03.680781 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.680785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.681265 (+ 480us) server_negotiation.cc:407] Connection header received
0807 08:34:03.681428 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.681431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.681474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.681515 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.681954 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51790: BlockingRecv error: recv got EOF from 127.0.0.1:51790 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 08:34:03.717531 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.716395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51802 (local address 127.1.157.125:38421)
0807 08:34:03.716506 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.716510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.716717 (+ 207us) server_negotiation.cc:407] Connection header received
0807 08:34:03.716842 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.716845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.716898 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.716986 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.717412 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51802: BlockingRecv error: recv got EOF from 127.0.0.1:51802 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 08:34:03.784516 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.783504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.1.157.125:38421)
0807 08:34:03.783640 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.783643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.783757 (+ 114us) server_negotiation.cc:407] Connection header received
0807 08:34:03.783891 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.783893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.783938 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.784005 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.784400 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51808: BlockingRecv error: recv got EOF from 127.0.0.1:51808 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 08:34:03.915261 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:03.914311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51812 (local address 127.1.157.125:38421)
0807 08:34:03.914438 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:03.914441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:03.914560 (+ 119us) server_negotiation.cc:407] Connection header received
0807 08:34:03.914681 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:03.914683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:03.914726 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:03.914797 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:03.915144 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51812: BlockingRecv error: recv got EOF from 127.0.0.1:51812 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 08:34:04.174151 3804 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 08:34:04.173215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.1.157.125:38421)
0807 08:34:04.173327 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.173332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.173431 (+ 99us) server_negotiation.cc:407] Connection header received
0807 08:34:04.173542 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.173544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.173589 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.173666 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 08:34:04.174028 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51828: BlockingRecv error: recv got EOF from 127.0.0.1:51828 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 08:34:04.503801 3944 catalog_manager.cc:1259] Loaded cluster ID: 660f32a204d04232b6d0736e30e400c7
I20260807 08:34:04.503873 3944 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: loading cluster ID for follower catalog manager: success
I20260807 08:34:04.505384 3967 catalog_manager.cc:1259] Loaded cluster ID: 660f32a204d04232b6d0736e30e400c7
I20260807 08:34:04.505447 3967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: loading cluster ID for follower catalog manager: success
I20260807 08:34:04.506043 3944 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: acquiring CA information for follower catalog manager: success
I20260807 08:34:04.507447 3944 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09caade5fe124056b05b38d737631fb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:34:04.507513 3967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: acquiring CA information for follower catalog manager: success
I20260807 08:34:04.508436 3967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd8206b8355e4169906516c48bf93f3c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 08:34:04.692092 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:04.688010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.1.157.125:38421)
0807 08:34:04.688236 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.688240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.688291 (+ 51us) server_negotiation.cc:407] Connection header received
0807 08:34:04.688404 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.688407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.688528 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.688624 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:04.688826 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.689403 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.690418 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.690694 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.690806 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:04.691119 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:04.691661 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:04.691773 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:04.691864 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 08:34:04.691919 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":67,"threads_started":1}
I20260807 08:34:04.698068 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:04.694826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.1.157.124:35075)
0807 08:34:04.695045 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.695049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.695077 (+ 28us) server_negotiation.cc:407] Connection header received
0807 08:34:04.695201 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.695204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.695302 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.695383 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:04.695564 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.696082 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.696909 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.697120 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.697198 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:04.697428 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:04.697765 (+ 337us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:04.697842 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:04.697914 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 08:34:04.697959 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":54,"threads_started":1}
I20260807 08:34:04.704241 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:04.700474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.1.157.125:38421)
0807 08:34:04.700624 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.700627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.701068 (+ 441us) server_negotiation.cc:407] Connection header received
0807 08:34:04.701223 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.701226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.701275 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.701342 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:04.701645 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.702393 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.703248 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.703482 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.703576 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:04.703788 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:04.703860 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:04.703922 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:04.704057 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 08:34:04.704112 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 08:34:04.704557 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:04.700528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.1.157.124:35075)
0807 08:34:04.700622 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.700626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.701058 (+ 432us) server_negotiation.cc:407] Connection header received
0807 08:34:04.701211 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.701215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.701256 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.701316 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:04.701513 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.702250 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.703444 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.703731 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.703830 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:04.704028 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:04.704109 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:04.704172 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:04.704396 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 08:34:04.704445 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 08:34:04.705361 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 08:34:04.700441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.1.157.126:33829)
0807 08:34:04.700999 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 08:34:04.701004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 08:34:04.701480 (+ 476us) server_negotiation.cc:407] Connection header received
0807 08:34:04.701570 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 08:34:04.701573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 08:34:04.701714 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 08:34:04.701806 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 08:34:04.702019 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.702725 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.703789 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 08:34:04.704106 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 08:34:04.704195 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 08:34:04.704578 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 08:34:04.704845 (+ 267us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 08:34:04.704921 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 08:34:04.705192 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 08:34:04.705252 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":57,"threads_started":1}
I20260807 08:34:04.707852 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3570
I20260807 08:34:04.719288 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3686
I20260807 08:34:04.728103 1653 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasksV4IkV/build/debug/bin/kudu with pid 3805
2026-08-07T08:34:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3126 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6229 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (280125 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST